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ABSTRACT 


Femtocells are low power base stations that communicate through a licensed spectrum 
with the intent to improve coverage and performance of voice and broadband services. 
The Femtocell works through a cellular network provider to enhance cellular 
portable/mobile devices especially in locations where coverage by cellular systems using 
large cells is weak and intermittent. 

The use of smartphones, tablets, and other wireless devices is becoming 
increasingly prevalent and is driving the need for innovations in wireless data 
technologies to provide more capacity, higher speed connections, and higher quality of 
service. Femtocells can provide a useful way for mobile operators to offer a better user 
experience and deliver broadband services indoors consistently and reliably for a 
comparable context of application, distances, and obstacles. 

In this thesis we will conduct a quantitative and qualitative analysis of Femtocell 
performance in comparison to that of Wi-Fi. Using COTS Femtocell and Wi-Fi 
technology an analysis will be conducted to establish which of the two is the better means 
of bringing internet connectivity to forward deployed forces. The potential benefits of 
this research are a better understanding of the advantages and disadvantages of Femtocell 
and Wi-Fi networks in simulated garrison and deployed environments. 
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I. 


INTRODUCTION 


A. BACKGROUND 

Femtocells are known by most as a type of home cellular base-station. 
Commercially, it has names like Network Extender (Verizon), Microcells (AT&T), and 
Airave (Sprint), among others, worldwide. It is a low-power base station that 
communicates through a licensed spectrum with the intent to improve indoor coverage 
and performance of voice and broadband services. The femtocell, working through a 
cellular network provider, enhances connectivity for cellular phones, smartphones, and 
other portable/mobile devices, especially in locations where coverage by cellular systems 
using large cells is weak and intermittent (i.e., indoors or remote areas). Ultimately, the 
user’s mobile devices are connected via the femtocell to a backbone network supplied by 
an Internet service provider. 

Although the femtocell architecture may seem different to the casual user, it is 
actually the same connection that a normal mobile device uses to access Internet 
connectivity. The difference is that a standard device connects through outdoor high 
power base stations and the femtocell is its own access point base station. This smaller 
localized base station provides very stable and efficient Internet connections. 

Femtocell network technology may seem like a new technology but early 
femtocell research was introduced in the late 1990s and has grown dramatically in the last 
decade. Both the general public and commercial mobile operators have shown increased 
interest in ways to improve upon and expand this technology through 3G and LTE/4G. 

Questions often asked are, “Why do we need femtocells when there is Wi-Fi 
technology,” and “Which one is better for the potential mobile device user?” The 
purpose of this thesis is to analyze femtocell and Wi-Fi network capabilities and 
performance to determine which is the better platform for military use in a potential 
tactical network. 
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B. PURPOSE 


The purpose of this thesis is to conduct an analysis of femtocell performance in 
comparison to that of Wi-Fi from the user’s point of view. Using commercial-off-the- 
shelf (COTS) femtocell and Wi-Fi technology, an analysis will be conducted to establish 
which of the two is the better means of bringing Internet connectivity to forward 
deployed military members. In order to accomplish this we will perform performance 
tests in the areas of Internet connectivity, uploading and downloading speeds, and Voice 
over IP (VoIP) in both ideal conditions and in realistic (less than ideal) conditions. The 
potential benefits of this research to the defense establishment are a better understanding 
of the advantages and disadvantages of femtocell and Wi-Fi networks in simulated 
garrison and deployed environments. 

C. SCOPE 

Our objective for this research is a better understanding and analysis of femtocell 
network performance, especially as compared to those of Wi-Fi. Our analysis will be 
based on several performance tests between a femtocell and a Wi-Fi in the areas of 
accessing the Internet, streaming data, and voice over IP (VoIP). 

In order to perform these tests we utilize a COTS Wi-Fi router and femtocell. The 
first series of tests will be baseline performance tests that will establish the basic 
performances in ideal conditions. We will then perform the same series of tests in a 
realistic environment (non-ideal situations). 

As mentioned earlier, the targeted areas of testing are accessing the Internet, 
streaming data, and VoIP. These areas were chosen because they are the most used 
methods of utilizing networks. Each of these areas can also be used by members of the 
military in both garrison and deployed environments. 

The first area is accessing the Internet through Hyper Text Transfer Protocol 
(HTTP). HTTP is the underlying protocol that is used by the World Wide Web. It 
defines the way in which messages are formatted and transmitted, as well as, what actions 
need to be taken. HTTP is used in every action of the process of accessing the web. 
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Real-time Transport Protocol (RTP) is the underlying protocol in our next area of 
evaluation: streaming data. RTP basically standardizes packet formats for delivering 
audio and video over IP networks. RPT is used for streaming media, teleconferencing, 
and real time data. 

The final area of testing is VoIP. VoIP is the process of transmitting voice traffic 
over IP-based networks. VoIP essentially compresses data packets during transmission 
which allows more data to be handled over the carrier. As a result VoIP can not only 
handle multiple callers at once, but it can also (through software applications) transmit 
video and data. 

As stated, the testing will be conducted in both ideal and non-ideal environments. 
Our definition of an ideal situation is one where we are located in the same room as the 
Wi-Fi router and Femtocell access point. The non-ideal environments consist of moving 
further away from the router and Femtocell, and include obstacles such as walls and 
floors. The number of users on the network is also included in our non-ideal 
environments. 

D. LITERATURE REVIEW 

Femtocells first came to light in 1996 through Silventoinen et al.’s, “Analysis of a 
new channel access method for home base-stations.” It described the potential of 
extending the concept of a home base-station. He described a simple architecture of a 
cellular network that used a peculiar Total Frequency Hopping. This basic idea led to 
suggestions of a requirement to double frequency re-use in both indoor and outdoor 
environments, years earlier covered in Kinoshita et al.’s “Frequency common use 
between indoor and cellular radio research on frequency channel doubly reused cellular 
system” (1989). From 1996 to present there has been significant research on femtocells. 
Joseph Boccuzzi et al.’s “Femtocell Design and Applications” (2011) and Jie Zhang et 
al.’s “Femtocells Technologies and Deployment” (2010) are some of the most recent and 
extensively cover the subject of femtocells with an emphasis on the deployment and use 
of the equipment in a more commercial manner. 
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The 3 ld Generation Partnership Project (3GPP), a partnership that produces 
technical specifications and reports pertinent to 3 generation cellular systems, has issued 
numerous releases that prepare and support the continuous evolution of femtocells (3GPP 
releases 9 and 10, TS 25.467 and 25.306, TR 21.905). In 2007, the Femto Forum was 
created to promote wide-scale adoption of femtocells. This forum has played a role in 
ensuring that the standards were agreed upon and released to the public. Publications 
from this forum, “Interference Management in UTMS Femtocells” (2010) and 
“Regulatory Aspects of Femtocells” (2011), speak to the challenges of interference and 
regulatory issues with which the femtocell community is currently dealing. There are 
also technical literature and periodic reviews that deal heavily with LTE prospects and 
transmission issues (V. Chandrasekhar et al.’s “Femtocell Networks: a Survey” (2008), 
and D. Knisely et al.’s “Standardization of Femtocells in 3GPP” and “Standardization of 
Femtocells in 3GPP2” (both in 2009). In 2010, methods to improve joint macro level and 
femtocell level frequency assignments and alternate optimized frequency reuse schemes 
are addressed in Y. Haddad et al.’s “Femtocell SINR Performance Evaluation” (2010). 
Additional challenges and issues relating to combining and synchronizing signals from 
other base stations are studied in S. Kim et al.’s “Performance Analysis of LTE 
Enterprise Femtocell Using Cooperative Downlink Transmission Scheme” (2011). This 
study stresses the need to utilize the LTE FDM scheme to get overlapping resources, 
which lead to better SINR and reinforced signals. 

Analyses of mixed Macro-cell and Femtocell cases are seen in B. Kaufman et 
al.’s “Femtocells in Cellular Radio Networks with Successive Interference 
Cancellation”(2011). These analyses look to introduce a Femtocell power control 
process that does not require coordination with macro-cells. The ultimate goal is defining 
an optimal Macro-cell-to-Femtocell hand-off. Other interference issues involved with 
Heterogeneous Networks (HeTNeT) are referenced in D. Lopez-Perez et al.’s “Enhanced 
Intercell Interference Coordination Challenges in Heterogeneous Networks” (2011). 
Lopez-Perez et al deals specifically with the control channel degradation problems and 
the application of different power control techniques in Femtocells. 
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Recently, numerous publications have been released dealing with the very 
important issues of resource assignments and optimization. A few examples of these are: 
G. de La Roche et al.’s “Selforganization for LTE enterprise femtocells” (2010), Y. 
Haddad et al.’s “Analysis of an Efficient Channel Assignment Scheme for Femtocell” 
(2011), F. Tariq et al.’s “Dynamic Fractional Frequency Reuse Based Hybrid Resource 
Management for Femtocell Networks” (2011), S. Das et al.’s “ Issues in Femtocell 
Deployment in Broadband OFDMA Networks: 3GPP-FTE a case study” (2011), and X. 
Chu et al.’s ’’Resource Allocation in Hybrid Macro/Femto Networks” (2010). 

D. de la Roche et al.’s “Selforganization for FTE enterprise femtocells” (2010) 
discusses methods and relating problems with the use of multiple FTE femtocells in 
different environments. De la Roche et al proposes that the best global throughput can be 
achieved by a specifically proposed self-organizing network technique. The need for 
Femtocell Access Points to share spectrum and several algorithms to share this spectrum 
across multiple Femtocell Access Points (FAPs) are explored in Y. Haddad et al.’s 
“Analysis of an Efficient Channel Assignment Scheme for Femtocell” (2011). Haddad et 
al states that a central database that holds information on all FAPs in most cases offers 
the better performance. Methods of using Hybrid Resource Management Algorithms 
(HRMA) for down-link OFDMA purposes in order to offer better performance with a 
larger number of nodes is discussed in F. Tariq et al.’s “Dynamic Fractional Frequency 
Reuse Based Hybrid Resource Management for Femtocell Networks” (2011). When 
referring to broadband, S. Das et al.’s “Issues in Femtocell Deployment in Broadband 
OFDMA Networks: 3GPP-FTE a case study” (2011) shows the benefits of the co¬ 
existence of macro/micro-cells and co-channel femtocells in OFDMA-FDD systems. 

Femtocell systems are addressed in a broader manner in numerous research 
papers. Some of the relevant publications are: S. Hassan et al.’s “Femtocell versus Wi-Fi 
-A Survey and Comparison of Architecture and Performances” (2009), F. Meshkati et 
al.’s “Mobility and Capacity Offload for 3G UMTS Femtocells” (2009), and M. Khan et 
al.’s “Focal IP Access (FIPA) Enabled 3G and 4G Femtocell Architectures” (2011). The 
latter describes several architectures for different FIPA scenarios. Trade off implications 
between capacity offload and UE battery life with regards to Femtocells in 3GPP UMTS 
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are the subject of F. Meshkati et al.’s, “ Mobility and Capacity Offload for 3G UMTS 
Femtocells” (2009). 

The similarities and differences between Femtocells and technologies such as Wi¬ 
Fi is the subject of S. Hassan et al.’s “Femtocell versus Wi-Fi-A Survey and Comparison 
of Architecture and Performances” (2009). This work states that both Femtocells and 
Wi-Fi can provide services based on the use of IP networks and that “evaluating their 
performances under the varied conditions of IP networks is an interesting area of future 
work.” 

E. THESIS ORGANIZATION 

Chapter I validated the need for this research by providing an overview of the 
purpose and relevance of this research. With its brief introduction and background to the 
subject of femtocells, Chapter I is intended to point out that there has been little research 
with regards to comparing femtocell and Wi-Fi capabilities and performance. The 
chapter also contains a literature review of relative femtocell research and concludes with 
a discussion of how this body of work is organized. 

Chapter II presents a brief history of femtocell technology. Chapter II goes on to 
list several issues and challenges associated with femtocell deployment. These included 
quality of service, frequency/bandwith, interference, handover, regulatory, and security 
challenges and issues. The chapter goes on to describe basic femtocell and Wi-Fi 
architectures. 

Chapter III presents a description of the methodology and experiments that will be 
conducted within this research. Chapter III began with establishing baseline testing in 
both ideal and non-ideal conditions. The chapter then goes on to describe testing in the 
areas of Internet accessing via HTTP, and downloading of various sized files, various 
sized RTP file streaming, and VoIP. 

Chapter IV lists, describes, and summarizes the data collected in Chapter III. 
Chapter IV goes on to point out the key findings and the results of the tests performed in 
Chapter III. 
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Chapter V provides an overall conclusion to the research study. The 
chapter revisits the intent of the research to ensure that all objectives set forth were 
adequately addressed. The chapter concludes by highlighting recommendations and 
potential future research topics relating to this research. 
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II. FEMTOCELL HISTORY, ISSUES, AND CHALLENGES 


A. A BRIEF HISTORY OF FEMTOCELLS 

The actual term femtocell is used to describe a coverage area, scale, or size. As 
shown in Figure 1, the macrocell is the largest level cell and it provides the widest range. 
The macrocell is found in most rural areas and can be located along major highways. 
The next smaller size cell is the microcell. It is used in very densely populated areas 
(mostly urban) like cities and large towns. Within these cells is the picocell, which is for 
areas that are even smaller. Picocells are often found in large office buildings, industrial 
areas, and commercial areas (i.e., shopping centers and malls). The smallest cell is the 
femtocell. Femtocells can be found in a person’s home or an individual office. 

Research into “small cells” can be found in literature as early as 1984. For 
instance, in his article “Small-Cell Mobile Phone Systems,” Arthur Stockton describes 
systems that have “direct access to the land telephone network and are designed to 
connect any mobile phone to any other phone, mobile or not.” In the 1990’s there was 
increasing demand for cellular services and as a result the macrocells were being 
overloaded. 



Figure 1. Macro-cell, micro-cell, pico-cell, and femto-cell ranges 
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This led to the development by Southwest Bell and Panasonic of a method of 
reusing the same frequencies as outdoor (macrocellular) cellular systems to provide 
wireless communications inside a building. This was accomplished by using a wired 
backhaul. Even though the technology wasn’t quite there to support the IP backhaul and 
it was very costly, it was the first actual femtocell type network. 

Over the last decade there has been a dramatic increase in consumer demand for 
increased capabilities through mobile means. According to Cisco Visual Networking 
Index (Cisco white paper: Forecast and methodology, 2011-2016), the amount of global 
mobile data traffic in 2011 has more than doubled for the fourth year in a row, and global 
mobile data traffic in 2011 was over eight times greater than the total global internet 
traffic in 2000. With this incredible growth, the need for new cellular architecture with 
greater capacity was necessary. Fortunately, the development of 4G standards that are 
based on Orthogonal Frequency-Division Multiple Access (OFDMA) and IP have 
provided a more efficient, low cost, platform for femtocells. 

Current technology has introduced automatic configuration and self-optimization 
capabilities in femtocells making them user friendly and, ultimately, sold in a plug-and- 
play type product. They also have the ability to automatically integrate into macro- 
cellular networks. As a result, over the last four years major femtocell deployments by 
the biggest cellular companies in the world has occurred. Sprint, Verizon, AT&T, and 
others worldwide now offer femtocells compatible with their underlying radio- 
infrastructures. 

B. FEMTOCELL ISSUES AND CHALLENGES 

1. Quality of Service Issues 

The term Quality of Service (QoS) refers to the requirements that are imposed by 
IEEE 802.11 on all aspects of an Internet connection. Some of these requirements are 
adequate signal-to-noise ratio (SNR), frequency responses, loudness levels, response 
time, loss, etc. The intent is to guarantee a standardized level of quality and performance 
for the consumer’s data flow needs. The issue with QoS for femtocells is that in order to 
achieve QoS requirements there often needs to be hardware changes. A possible solution 
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may be to use a traffic classifying service like Differentiated Services (DiffServ) or 
Integrated Services (IntServ). IntServ improves QoS by having applications use resource 
reservation protocol (RSVP) to improve requests and reserve resources through a 
network. DiffServ prioritizes packets according to the type of service they desire. 
Routers and switches can prioritize these to improve quality. WiFi, however, already 
must comply with IEEE 802.1 lQoS standards, and currently has established mechanisms 
in place to ensure QoS. 

2. Frequency / Bandwidth Issues 

The electromagnetic spectrum is a scarce and crowded resource. Femtocells 
operate on the same licensed spectrum that is allocated to cellular service providers. To 
deal with this overcrowding issue two methods have been used: the Co-channel 
Frequency Deployment and Orthogonal Channel Deployment. The Co-channel 
Frequency Deployment simply allows the femtocell and the cellular macro-cell to use the 
same frequency band. With co-channel use, however, there are identified interference 
issues. Orthogonal Channel Deployment is in many ways the opposite of Co-channel 
Frequency Deployment. In this method macro-cells and femtocells use separate 
channels. The advantage to this method is that there is less potential for interference, the 
disadvantage is a reduction in the overall system capacity. 

WiFi networks use different Industrial, Scientific, and Medical (ISM) frequency 
bands. These unlicensed ISM bands are operated independently of any specific cellular 
service and are available for public use. This however may also lead to interference 
problems when too many WiFi devices are located near each other using the same band. 

3. Interference Issues 

As stated earlier, there is limited spectrum on which cellular systems can operate 

and the spectrum is controlled by licensing. Femtocells utilize the spectrum already 

licensed for cellular providers. Thus, interference is a key issue associated with 

femtocells. When multiple femtocell devices are being serviced by the same macro-cell 

there can be adjacent channel interference. There can also be interference issues when 

several femtocell devices are used in close proximity to each other, regardless of whether 
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or not they are serviced by the same macro-cell. Generally though, femtocells are used in 
areas of poor or limited cellular coverage and in these cases interference from 
overcrowded networks is not an issue. Also, a benefit of the low power output of the 
femtocell is that multiple femtocell devices would have to be very close to each other to 
cause interference. WiFi devices can also face similar interference issues stemming from 
the fact that all the WiFi devices are working on the same unlicensed band, are often in 
the same vicinity as other WiFi access points or user devices, and use a very limited 
number of non-overlapping channels - specifically, three of the eleven available in the 
U.S. (of the twelve overseas). This latter fact impacts the utility of WiFi in congested 
areas. 


4. Handover Challenges 

When a mobile device in a WiFi network moves to the outer edge of its Received 
Signal Strength (RSS) limit it needs to perform a “handover” of connection from one 
access point to another. The major concern for femtocell handover is that the coverage 
area of an individual femtocell is very small. For this reason, it becomes essential that 
there is a seamless handover to and from femtocells so the user can maintain continuous 
signal connectivity. There are generally three types of handovers for both WiFi and 
Femtocells. The first is a simple base station to base station handover where a user 
moves from the range of one base station to another. The second occurs between base 
stations and Femto Access Points (FAPs). 

The base station to FAP handover happens when the mobile user moves from an 
outdoor area to an indoor area. When the user starts outdoors it sends a request to a 
cellular base station and when the user then moves indoors the FAP will accept the 
request and pick up the signal. For this to work there has to be synchronization between 
the FAP and the cellular base station. 

The final handover scenario is where the user moves from one FAP to another. 
This generally happens when there are multiple FAPs in the same vicinity, in an office 
building for example. The challenge associated with handovers for femtocells is that they 
are not usually connected to a network environment where mobility is addressed, (again 
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as in an office building where mobility outside the building isn’t a concern). Due to the 
fact that the femtocell must be associated with an IP address, whe a user is mobile the IP 
addresses would have to change. 

5. Regulatory Challenges 

One of the biggest differences between WiFi and femtocells is the fact that WiFi 
operates in an unlicensed spectrum while femtocells operate in a licensed spectrum and 
require regulatory approval. This becomes an issue because the spectrum and radio 
regulations will vary from one country to the next. International agreements can also be 
involved when a user takes their femtocell from one nation to another. In a licensed 
spectrum the provider pays substantial sums to be able to use a portion of the spectrum 
exclusively and regulators will enforce transgressions. This means that a femtocell 
operator could not just move their femtocell to another country and operate it. The 
varying spectrum allocations from one country to another can also prevent unauthorized 
usage. 

A femtocell has several means to identify where it is. The first is a Global 
Positioning System (GPS) receiver that is built into the femtocell. This immediately 
identifies the location of the femtocell. Another means is by mapping its IP address to 
the femtocell’s originating country. A femtocell also can sense other cell site identities in 
its area and can identify its “neighborhood.” If a femtocell sees that it is in an 
unauthorized area it can disable itself or notify the provider. 

Due to the regulatory issues operators cannot use their femtocells in frequency 
spectrum that they do not own and control. Some large providers (Verizon, AT&T, T- 
Mobile) may have operations in several different countries and therefore they license the 
spectrum in those countries. A femtocell user with one of these companies would still 
not be able to use their femtocell in these countries because the spectrum allocation may 
be different and the femtocell would still broadcast its original identity (trying to connect 
to its home network). Many femtocells have 2G and 3G receivers that can scan for 
signals from external cellsites and can determine the country in which it is located and 
what networks are available. The surrounding cellsite identities will change if the 
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femtocell is moved. This could interfere with local mobile phone users who could 
possibly pick up the signal, and would also cause unnecessary hand-offs that would 
reduce signal strength. 

6. Security Challenges 

The security of a device or network is always a paramount concern for users, 
especially on a wireless medium. There are three major security vulnerability concerns 
for femtocell network technology. The first comes from the wireless link into the 
femtocell. According to a technical white paper from Picochip (2011), it is possible for 
external wireless transmissions to potentially gain unauthorized access to the femtocell. 
The second concern is the backhaul link that is used between the femtocell and the 
gateway into the service provider’s core network (the Internet link). The third concern is 
the femtocell itself, as it is potentially possible for nefarious network users to get into the 
femtocell and take control of it remotely. 

There are several ways to prevent or counter these security issues. The first is to 
ensure secure authentication. Authentication needs to be required by the service provider 
or the operator to correctly identify valid femtocells within the network. Another means 
of ensuring security is the use of Internet Protocol Security (IPsec). IPsec is a protocol 
for securing IP communications by authenticating and encrypting each IP packet. It also 
establishes mutual authentication and provides cryptographic keys for additional security. 
Extensible Authentication Protocol (EAP) is an authentication framework for wireless 
networks and also provides a means of ensuring wireless security. 

C. BASIC FEMTOCELL ARCHITECTURE 

A basic femtocell network architecture, as shown in Figure 2, is generally 
comprised of three elements: a Femtocell Access Point (FAP), a security gateway, and a 
femtocell management system. The FAP base station also requires a means of 
connecting to the Internet, typically through a broadband Internet connection (DSF, cable 
modem, or direct ISP access). 

The Femtocell Access Point is basically a small scale cellular base station. 

It is the primary node in the network that connects the user to the network, and can be 
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used in stand-alone or integrated configurations. When used in the stand-alone 
configuration the FAP is connected directly to the user’s router, whereas in an integrated 
method the FAP has its own built-in router. A typical femtocell access point base station 
will transmit 100 mW of power, has a data rate of between 7.2 and 14.4 Mbps, operates 
at between 1.9 and 2.6 GHz, and has an effective range for high performance of 
approximately 100 feet (dependent upon location and interference issues). Note that this 
data rate is comparable to IEEE 802.3 10BASET connections (traditional “Ethernet”). 



Figure 2. Basic femtocell architecture 


In all systems pertinent to operations and support, security is a very important 
issue. In a femtocell architecture the security gateway is a network node that provides a 
secure means to access the Internet. The security gateway uses Internet Protocol Security 
(IPSec) and Internet Key Exchange (IKEv2) Internet security protocols for encryption 
support and for the authentication and authorization of the femtocells. Femtocell 
security gateways are network nodes that have been designed for use in carrier-type 
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networks and meet availability, scalability, and network management security 
requirements. 

The femtocell management system is arguably the most important element in the 
femtocell architecture. A femtocell management system must comply with Technical 
Report 069 (TR-069), which is a protocol for communication between Customer Premise 
Equipment (CPE) and Auto-Configuration Servers (ACS) that encompasses secure auto¬ 
configuration as well as other CPE management functions within a common framework. 
The femtocell management system plays a critical role in the operational management, 
provisioning, and activation of the femtocells. It is the femtocell management system 
that allows the operator to control the device remotely ensuring that it is in compliance 
with local regulations. 

For Internet connectivity, femtocells connect to the mobile operator’s network via 
a standard broadband connection, such as DSL, fiber, or cable. The data to and from the 
femtocell is carried over an IP technology-based network provided by an Internet Service 
Provider. For wireless (mobile device) users, the connection to the femtocell is done via 
the normal cellular service technologies just as if they were using a conventional macro- 
cellular network to connect. 

D. BASIC WI-FI ARCHITECTURE 

A wireless local area network (WLAN) is a collection of wireless devices that 
will maintain connectivity with each other while transferring data. The WLAN works in 
three basic configurations: peer-to-peer, bridge, and wireless distribution system. Peer- 
to-peer configuration is where each computer in the network can act as a client or server 
for the other computers in the network. This allows them shared access to files (such as 
audio, video, data, etc.) and peripherals without needing a central server. A bridge 
configuration is used to connect networks. This is done by use of a wireless Ethernet 
bridge, providing the connection for devices to a wireless network. The wireless 
distribution system enables the wireless inter-connection of the access points within a 
network. This allows a wireless network to be expanded through the use of multiple 
access points linked together. Generally, a WLAN’s signal can reach to 500 feet indoors 
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and approximately 1000 feet outdoors. WiFi operates in the unlicensed 2.4 GHz or 5.8 
GHz ISM Band. WiFi transmissions are essentially FM transmission, in that the 
frequency is changed to transmit data. The 2.4 GHz spectrum is shown in Figure 3. 



Figure 3. 2.4 GHz Wi-Fi channels from Gauther, M., Wireless networking in the 

developing world (2009) 

WiFi is a limited range wireless networking protocol based on the IEEE 802.11 
standards. Having WiFi connectivity allows a user to transfer data at the speed of 
broadband using radio waves rather than a wired or cabled infrastructure. A short-range 
wireless network (often referred to as a WiFi network or Wireless Local Area Network 
(WLAN)) is set up by using radio signal frequency to communicate among computers 
and other wireless-enabled devices. The main architectural components of a wireless 
network are the wireless router (access point), WiFi cards, safeguards, and one or more 
wireless clients. In simplest terms, an Access Point (AP) is a wireless LAN transceiver, 
or “base station,” that can connect one or many wireless devices simultaneously to the 
Internet. WiFi cards are installed in client devices and accept the wireless signal and 
relay information. Safeguards are firewalls or anti-virus software products that protect 
networks and help to keep information secure. 

As shown in Figure 4, the basic WiFi architecture starts with a station. This is 

essentially a computer that can be either mobile or fixed. A Basic Service Set (BSS) is 

created when two or more stations come together in order to communicate with each 

other. According to IEEE 802.11, there are two types of operating modes: infrastructure 
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mode, and ad hoc mode. Infrastructure mode is used to connect a computer with a 
wireless network adapter (or wireless client) to a wired network. This is accomplished 
through a wireless router or access point. Ad hoc mode is used to connect wireless 
clients directly together. This method does not use a wireless router or access point. The 
ad-hoc network refers to when a BSS is not connected to an Internet interface device and 
it is then referred to as an independent BSS (IBSS). 

When two or more basic service sets need to be connected it is done through a 
Distribution System (DS). The DS increases network coverage by allowing the wireless 
network to be expanded using multiple access points without needing a wired backbone 
to link them. 



Figure 4. Basic Wi-Fi network 


In this chapter we discussed the functionality of femtocells, and introduced the 
reader to some of the prevalent issues and challenges of femtocell technology. These 
items need to be resolved for the femtocell to be an accepted and widely used technology. 
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The key is to ensure that femtocells are scalable, and easily integrated. Femtocells also 
must be robust enough to successfully deal with security, regulatory, and interference 
issues. This chapter ended with a basic description of both femtocell and WiFi 
architectures. With a basic understanding of the history, issues, and architecture of 
femtocells we now turn to the purpose of this work, which is to compare and evaluate the 
performances between femtocells and WiFi networks. 
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III. METHODOLOGY 


A. INTRODUCTION 

The approach to this work is directly linked to the purpose, objectives, and scope 
listed in Chapter I. There is currently very little research published specifically 
developing an analysis of Femtocell versus WiFi in terms of their respective performance 
and capabilities. In this chapter, we provide the methodology for the tests and extensive 
experimentation conducted with respect to these technologies. 

Our research includes numerous testing tools that provide multiple measurements. 
These are evaluated and compared as to suitability for the experiments conducted herein. 
To compare the performances between Femtocell and WiFi, we must start first with a 
stable environment where the testing can be repeated with no outside interference nor 
variation. In our study, we use a single laptop computer that functions as a client. The 
laptop is alternately connected to the Internet through either a router and a Femtocell or a 
wireless router alone. When using the Femtocell, the laptop is connected to the Internet 
via a mobile hotspot provided by a cellular “smart” phone. 

Femtocells are base stations that, by design, connect automatically to a cellular 
mobile operator network. This internal connection process renders it impossible to 
manually create and manipulate your own network. Thus, due to its server-client 
architecture, communication among devices directly through the femtocell is not possible 
and can only be accomplished through the transfer of data from the client, to the network, 
and through the server. Due to these facts, we chose to use a single laptop computer as a 
client with both access to a Wi-Fi router and a mobile hotspot. As this is how a mobile 
phone accesses the Internet this is actually beneficial for our research, as it provides a 
more realistic environment for testing. 

B. TECHNICAL SPECIFICATIONS 

The devices used in this research can be described as micro-environmental 
devices. This means that they are all located indoors and are purchased, owned and 
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operated by the average Internet user. They include a wireless router (802.11 wireless 
access point), a Femtocell (Verizon Network Extender), a Motorola Droid RAZR 4G 
cellular phone (with its mobile hotspot enabled), and network endpoints (client and server 
nodes). A comparison of the general specifications for Wi-Fi and Femtocells can be seen 
in Table 1. 



Wifi 

Femtocell 

Data Range Capabilities 

11 and 54 Mbps 

7.2 - 14.4 Mbps 

Operating Frequency 

2.4 and 5 GHz 

1.9-2.6 GHz 

Power Output 

100, 200 mW 

10, 100 mW 

Range 

100 - 200 m 

20 -30 m 

Services Provided 

Voice & Data 

Voice & Data 


Table 1. General Wi-Fi femtocell specifications 


1. Wireless Router 

A wireless router is a device that provides wireless signals for connecting network 
devices that have wireless adapters (Table 2). The purpose of the wireless router is to 
send wireless signals that can be interpreted by the wireless-enabled network clients for 
communicating data and information. Routers collect signals and convert them into 
wired signals and send them over the Focal Area Network (FAN). A wireless router will 
generally have a 20 - 30 meter indoor range and approximately 80 - 100 meter outdoor 
range. Routers can usually connect several wireless devices within their area of 
coverage. 
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Model 

Netuear WGR614 fv7) 

Frequencies: 

2.4 - 2.5 GHz 

Network Speed/Data Rates: 

54 Mbps 

Antenna: 

2 dBi 

Encryption: 

64 bit, 128 bit, and 152 bit, WEP encryption, 
WPA (WiFi Protected Access) 

Data and Routing Protocols: 

TCP/IP, RIP-1, RIP-2, DHCP, PPP over 
Ethernet (PPPoE) 

Interface Specifications: 

LAN: 10BASE-T or 100BASE-TX, RJ-45 

WAN: 10BASE-T or 100BASE-TX, RJ-45 

Data Encoding: 

802.11b: Direct Sequence Spread Spectrum 
(DSSS) 

802. llg: Orthogonal Frequency Division 
Multiplexing (OFDM) 


Table 2. Wireless router specifications from Netgear Support www.Netgear.com 


2. Femtocell 

The femtocell is a small cellular base station that is designed for use in small 
areas, like homes and buildings (Table 2). It works by connecting to a service provider’s 
network through broadband networks (DSL or cable). Femtocells have a range of 20 - 
30 meters, and can generally support up to six devices. A comparison of all 
commercially available Femtocells is provided in Figure 5. For our testing we will be 
using the Samsung Wireless Network Extender provided by Verizon (Table 3) connected 
directly to the wireless router via an Ethernet cable. 
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Carrier 

AT&T 

Verizon 

Sprint 

T-Mobile 

Solution 

Femtocell - "3G 

MicroCell" 

Femtocell 

"Network 

Extender" 

Femtocell 

"Airave" 

UMA 

"HotSpot @ Home" 

Branding 

Cisco 

Samsung 

Samsung 

NA 

Technology 

3G UMTS/HSPA 
for voice and data 

2.5G CDMA 

2000 lxRTT 

2.5G CDMA 
2000 lxRTT 

UMA voice over 
WiFi 

Simultaneous 

Calls 

4 Simultaneous 

3 Simultaneous 

3 Simultaneous 

NA 

Standby 

Approved 

Callers 

10 

100 

50 

NA 

Data Bitrate 

3.6 megabits/s 

(HSDPA 3.6) 

144 kilobits/s 

144 kilobits/s 

NA 

GPS Fix 

Required 

Yes 

Yes 

Yes 

NA 

Hand-On/Hand- 

Off 

No/Yes 

No/Yes 

No/Yes 

Inter AP 

Handover/Yes 

Coverage 

5000 square feet 

5000 square feet 

5000 square feet 

WiFi AP range 


Figure 5. Femtocells; A comprehensive exploration, www.anandtech.com, from 

Brian Klug (4/1/2010) 


Model 

Samsung Wireless Network Extender (SCS- 


2U01) 

Frequencies: 

800/1900 MHz 

Air Interface: 

CDMA2000 lx Rel 0 

CDMA2000 EvDO 0/A 

Traffic Channel: 

Up to six simultaneous users (a seventh is 
reserved for emergency calls) 

Transmission: 

10/100 Base-T Ethernet/Network 

Standards: 

IEEE 802.3, IEEE 802.3u for Ethernet 

IEEE 802.11 g, IEEE 802.11b for Wireless 

Power Range: 

10 mW to 30 mW 


Table 3. Samsung wireless network extender femtocell specifications 
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3. 


Cellular Smartphone 


In this research, we use a Motorola Droid RAZR cellular phone with the Mobile 
Hotspot application enabled with the specifications found in Table 4. The Motorola 
Droid RAZR is a 4G capable LTE smartphone that has a built in Mobile Hotspot 
application. A mobile hotspot is a means of allowing the mobile phone to act as a Wi-Fi 
access point, providing a network access to nearby computers. Tablets and other Wi-Fi 
capable devices. The Motorola Droids mobile hotspot application allows the phone to 
connect to a mobile data network and then act as a Wi-Fi router, distributing the 
bandwidth to nearby clients. Any Wi-Fi enabled computer or mobile device can connect 
to the network that the mobile hotspot provides. A significant advantage to the mobile 
hotspot provided by the Droid RAZR is that it is completely mobile. The cellular phone 
receives its signal through the nearest macro-cell station. Using this method, however, 
requires the user is in the provider’s area of coverage. The connection speed provided by 
the mobile hotspot depends on many variables, including the cell network to which you 
are connecting, how far you are from the network’s closest transmission tower and how 
congested the shared service may be at the time. A typical cellular phone mobile hotspot 


has a range of approximately 100 feet. 


Model 

Motorola Droid RAZR XT912 

General: 

2G GSM 850/900/1800/1900 CDMA 800/1900 

3G HSDPA 850/900/1900/2100 CDMA2000 lxEV- 
DO 

4G FTE 700 MHz Class 13-For Verizon 

Memory: 

16 GB storage, 1 GB RAM 

Data: 

GPRS: Class 12, 32-48 kbps 

EDGE: Class 12 

Speed: Rev. A, up to 3.1 Mbps, LTE, HSDPA, 
HSUPA 

WLAN: Wi-Fi 802.11 b/g/n, DLNA, Wi-Fi hotspot 
Bluetooth: v4.0 with LE+EDR 

Operating System: 

Andoid OS, v4.0.4 (Ice Cream Sandwich) 

Browser: 

HTML, Adobe Flash 

CPU: 

Dual-core 1.2 GHz Cortex-A9 

Chipset: 

TI OMPA 4430 

Java: 

Via Java MIDP emulator 

GPS: 

With A-GPS support 


Table 4. Droid RA Z R XT912 specifications 
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4. 


Network Endpoint 


A network endpoint is a device that enables a user to access network services. In 
this research we use the Motorola Droid RAZR and a Hewlett-Packard laptop computer 
as seen in Table 5. The Droid RAZR will create a mobile hotspot, and the HP laptop will 
alternately utilize the created hotspot network and the router created Wi-Fi networks to 
run the tests needed to measure and compare the performances and capabilities of the two 
technologies. 


Model: 

Hewlett-Packard HP G62 Notebook PC 

Microprocessor: 

2.50GHz VISION Technology from AMD with AMD 
Turion II Dual-Core Mobile Processor N530 

Memory: 

4GB DDR3 System Memory (2 DIMM) 

8MB (max memory) 

Video Graphics: 

ATI Mobility Radeon HD 4250 Graphics 

Video Memory: 

Up to 1917 MB 

Hard Drive: 

320 GB (5400RPM) 

Network Card: 

Integrated 10/100 Ethernet LAN 

Wireless Connectivity: 

802.11 b/g/n WLAN 

Operating System: 

Windows 7 Home Premium 64-bit 


Table 5. Network endpoint specifications 


C. TESTING 

The ultimate goal of this research is to evaluate the utility of Femtocells by 
comparing their performance to that of a traditional wireless Wi-Fi network. To 
accomplish this, our research evaluated the performance and capabilities of these two 
technologies through several use-cases, applications, and scenarios. Our research 
performed baseline tests and assessment of the basic performances of these two 
technologies in ideal conditions. These tests are described below. We then transitioned to 
a realistic environment evaluation. The realistic environment tests were conducted in less 
than ideal settings that involve both line-of-sight obstacles and distance from the access 
point or femtocell. Support for Internet applications, such as browsing or “surfing” 
through HTTP protocol, RTP protocol applications for streaming, and transfer of 
audio/video files was also conducted. Finally, we evaluated the VoIP supporting 

protocols offered by both Femtocell and Wi-Fi provisioned networks. 
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1. Testing terms 

Ping is basically the process of sending an echo-request packet from the user’s 
computer to a different or remote computer (or server). The time between the 
transmission of the request (or ping) and the receipt of the associated echo-reply is a 
measurement of the latency of the connection. Ping is measured in milliseconds (ms). If 
the user experiences a delayed response in Internet applications it could be due to a 
higher than desired network latency. Latency is a term that basically means the delay 
during the performance of a given operation. Latency is used to describe any type of 
delay that occurs during the transmission or processing of data packets, such as 
transmission, propagation, processing, or queuing. 

Jitter is the variance in measuring successive ping tests. A reading of zero in a 
jitter test means that the results were exactly the same every time. A score above zero 
indicates the amount by which they varied. The lower the jitter value the better the 
connection service quality for applications sensitive to delay. 

Packet Loss is the term used to refer to unsuccessful transmission of “packets” of 
data. Having packet losses usually means that there is a deficiency associated with your 
Internet connection. Losses of packets may reduce upload and download efficiency, 
particularly due to requirements for retransmissions by applications sensitive to packet 
loss, lead to poor quality VoIP audio, and pauses in streaming media. Packet loss, 
generally associated with network congestion and its inherent packet collisions on 
wireless links or queue-overloads, is a metric where anything greater than zero percent 
may be an issue. 

Packet Order is a measure in percentage of how many packets arrived in order. 
Packets do not necessarily take the same route or the same time to reach their 
destinations. This results in packets arriving out of order, which causes other packets to 
be delayed or discarded. Delayed or discarded packets may cause a performance problem 
for the application, and as noted above, may lead to increased retransmissions which 
exacerbate the network performance issues. 
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Packet Discards is a measure of packets that arrive too late to be used by the 
application. Packet arrivals may be very time sensitive, especially with respect to media- 
based applications, such as audio or steaming-video. If a packet arrives too late the 
application performance suffers, and the packet has to be intentionally discarded, 
effectively wasting the network resources used to deliver it. 

A Mean Opinion Score (MOS) is a measure from 1 (being the worst) to 5 (being 
the best) as a rough order of service quality. MOS originated from the phone companies 
and used human input from related quality tests. Software applications have adopted the 
MOS score and scale. MOS scoring can be described as follows: 5 - Clear, as if in a real 
face-to-face conversation; 4 - Fair, small interference but sound is still clear; 3 - Not fair, 
enough interference to start to annoy the user; 2 - Poor, very annoying and almost 
unusable; 1 - Not fit for purpose. 

Downloacl/Upload Speed- more appropriately referred to as rate - is a 
measurement of how fast a user’s connection can deliver content to/from their computer. 
Note that this is generally a relative measure and not the theoretical value for the link. It 
must also be specified whether the value refers to the consolidated rate for the link or the 
effective rate for individual hosts. For example, the upload speed of a satellite link may 
be 1.5 Mbps; however, that capacity is shared among all users accessing that link. Thus, 
if 20 users are concurrently accessing the link, each may only receive 75 Kbps of service. 
When collecting data pertinent to upload or download rates, one must be cognizant of the 
user population. 

To achieve the optimal delivery of information for applications like VoIP, email, 
and on-line interactive programs, the receiving party’s download rate must be at least as 
fast as the sending party’s upload rate. In most cases uploading files is slower than 
downloading files. This is due to the fact that most Internet connection devices are 
asymmetrical. This means that they are designed to provide better downloading rates 
than upload rates. The reason for this is that most users spend the majority of their time 
on the Internet viewing web pages or using multimedia files which involve downloading. 
For this reason, the average uploading rate is typically much slower than the average 
downloading rate. 
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Round Trip Time is the time it takes for a packet to be sent end-to-end between 
the client and the server and for a response to be received back from the recipient. A 
long round trip time will dramatically slow connection throughput performance, 
particularly for TCP-based applications, and an erratic round trip time is an early 
indication of congestion problems. 

2. Baseline Testing 

The baseline tests performed in this research were conducted to establish 
preliminary network parameters in ideal settings. These tests addressed parameters such 
as: bandwidth, download and upload rates, packet losses and transfers, signal strength, 
ping, and jitter. These tests were performed first to ascertain the best-case values that are 
achievable. These tests also identify where major differences between Femtocells and 
Wi-Fi exist. The assumptions we make for the baseline testing are, first, that we have an 
ideal channel that does not have any imperfections, interference, or delays; and, second, 
that the transmission and reception of the data takes place within normal traffic 
conditions. 

To perform the baseline tests we utilized three open-source software tools. These 
included Pingtest.net, Speedtest.net, and Ping-test.net. Pingtest.net is an online 
performance-measuring tool that determines the quality of the user’s broadband Internet 
connection with respect to latency. It does this through the measuring of round-trip-time, 
jitter, and packet loss. This tool also gives an overall grade of the user’s broadband 
quality. Speedtest.net is also an online performance-measuring tool that tests the users 
Internet connection bandwidth with respect to upload and download capabilities between 
the assessed client and a remote server hosting the Speedtest application. Finally, Ping- 
test.net tests the performance of a user’s Internet connection by checking how fast the 
user can download and upload data. It accomplishes this by sending both large and small 
packets of information through the Internet connection and measuring the speeds of their 
travel. This tool also addresses latency by measuring the round-trip-time time. 
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3. HTTP Web Access Testing 

Hypertext Transfer Protocol (HTTP) is a stateless, application-layer protocol used 
to transfer data on the Internet. Web browsers and servers exchange information in 
accordance with the rules of HTTP. HTTP is a request/response protocol, which means 
that a web browser will initiate a request to a server and the server in turn sends a 
response, thus providing an information “pull” service. HTTP is used for every web page 
access and is used in every action involved in Internet “web surfing.” 

As HTTP is the underlying protocol used by the World Wide Web, it is very 
important to our research. Our testing addresses HTTP downloading and points out the 
differences between the capabilities of Wi-Fi and cellular in terms of downloading data 
from the Internet. We also consider the differences between the way femtocell and Wi-Fi 
access the Internet. 

To address these areas, we use an open source tool entitled “Downtester.” 
Downtester assesses Internet download speeds from multiple locations throughout the 
world. It allows the user to choose URLs and systematically tests the download speed of 
each. For this research, we chose the following six URLs: http://www.google.com, 
http://www.facebook.com, http://www.yahoo.com, http://www.baidu.com, 

http://www.youtube.com, and http://www.cnn.com. We also use Downtester to measure 
the time to download files of varying sizes. We specified test files that are 20 MB, 50MB, 
100MB, and 200MB in size. We hold that these sizes represent the average users 
download needs. A 20MB file is considered a small file, equivalent to a standard-quality 
movie trailer download. A 50MB file is considered a medium sized file, equivalent to an 
MP3 audio CD download. The 100MB file is also considered a medium sized file, but it 
would equate to a high-quality MP3 audio CD download or a 2 minute high definition 
movie trailer. The 200MB file is a large file that would represent approximately 45 
minutes of a video stream or a large operating system update. 

Finally, we also use HTTP Analyzer, which is a tool that allows the user to 
monitor, trace, and analyze HTTP traffic in real-time. 
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4. RTP Streaming Testing 

To test streaming capabilities we utilize Real-time Transport Protocol (RTP) and 
Real-time Transport Control Protocol (RTCP). These protocols provide end-to-end 
network transport functions for applications transmitting real-time data, such as 
interactive audio and video. Using RTP/RTCP, we can analyze videos of different sizes 
by using a packet sniffer program. Utilizing a packet sniffer program and a streaming 
video player, we can analyze the stream for information, such as sequence errors, jitter, 
packet losses, etc. This will provide a comparison between the performances of 
Femtocells and Wi-Fi for streaming video content. 

To perform these tests we use the packet analyzing freeware program Wireshark. 
We use various size video files (20MB, 50MB, and 100MB) to stream through the 
VideoLAN Client (VLC) player. While streaming, we use Wireshark to capture packets 
for analysis. 

5. VoIP Tests 

Voice Over IP (VoIP) is voice communications delivered using Internet Protocol. 
This means sending voice information in digital form in discrete packets over packet- 
switched networks rather than the traditional switched-circuit sessions used in Public 
Switched Telephone Networks (PSTN). In these tests we analyze the differences 
between a Wi-Fi based connection and a Femtocell based connection. It is generally 
accepted that the better the upload/download speed the better the connection during a 
VoIP call. In the case of VoIP, jitter and packet loss are also factors for good quality. 

To perform these tests we use two open-source online testing programs, 
VoIPreview.org/voipspeedtester and myspeed.visualware.com. VoIPreview is an online 
program that evaluates the parameters of a VoIP phone call. Myspeed.visualware.com is 
an online program that provides the calculations of all parameters necessary for a 
successful VoIP call and establishes the value for an all-encompassing Mean Opinion 
Score (MOS) this is shown in Table 6. This score is a quality-based score as introduced 
above. These values are not always presented in whole numbers; certain limits or 
thresholds are expressed in decimal form. The range of 4.0 to 4.5 is generally accepted 
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as the level that provides a quality VoIP call. In contrast, values below 3.5 are 
considered unacceptable. 


1 

Impossible to communicate. 

2 

Very annoying. Nearly impossible to communicate. 

3 

Annoying 

4 

Fair. Imperfections can be perceived, but sound still 
clear. This is the range for cellular phones. 

5 

Perfect. Like face-to-face conversation or radio reception. 


Table 6. Mean Opinion Score (MOS) 


D. TESTING CONFIGURATIONS 

In order to provide the viable comparison between Femtocells and Wi-Fi in terms 
of their respective capabilities and performance, we must examine how they function in 
two different configurations. Due to the fact that Femtocell devices cannot provide 
Internet connectivity by themselves, ad hoc networks needed to be created. In our 
research two ad hoc networks were created for testing and comparison purposes. The 
first configuration is the traditional wireless network consisting of a laptop accessing the 
Internet via a wireless router (Figure6). This configuration is then compared to the 
second configuration that utilizes the Femtocell. In this second configuration, the 
Femtocell utilizes the routers internal Ethernet switch to provide Internet connectivity to 
the Motorola Droid RAZR cellular phone (Figure 7). The Droid in turn, provides a 
mobile hotspot that allows the wireless laptop to access the Internet. 



In this configuration, tests performed will be conducted with a traditional wireless 
network utilizing a wireless laptop. 
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Figure 7. Internet to wireless router to femtocell to cellular mobile hotspot to wireless 

laptop 


In this configuration, the Femtocell utilizes the wireless router (via an Ethernet 
cable) to provide Internet connectivity to the Motorola Droid RAZR. The Droid RAZR 
uses this to then provide a mobile hotspot that is used by the wireless laptop to access the 
Internet. 

In this chapter we have provided the methodology for the testing and extensive 
experimentation that we will be conducting with respect to these technologies. We 
discussed the technical specifications of our devices, and briefly discussed the applicable 
testing terms that will be used during our testing. We have also described each of the 
areas where we will be testing the capabilities and performance of the Femtocell and the 
Wi-Fi devices. In our next chapter, we will begin our testing. We will first establish a 
baseline evaluation of both devices and utilize this information to compare the results of 
further testing in the areas of HTTP web accessing, RTP video streaming, and VoIP 
applications. 
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IV. TESTING AND RESULTS OF TESTING 


A. BASELINE TESTING 

1. Introduction 

The baseline tests performed in this research were conducted to establish 
preliminary network parameters in near ideal settings. These tests address parameters 
such as: bandwidth, download and upload speeds, packet losses and transfers, signal 
strength, round-trip-time (RTT), and jitter. These tests must be performed first to 
ascertain the best case values that are achievable by each respective technology. These 
tests will also identify where the major differences between Femtocells and Wi-Fi exist. 
In this testing, we must assume that we have an ideal channel that does not have any 
imperfections, interference, or delays. We must also assume that the transmission and 
reception of the data takes place within normal traffic conditions. Testing under field-like 
conditions followed the baseline tests. Performance was measured using the three on-line 
tools described in Chapter 3. These included speedtest.net ping-test.net, and pingtest.net. 
The results of the tests are presented below. 

2. Baseline Test Evaluation 

The baseline tests results are relatively consistent in each of the respective tests. 
The averages are also what we expected from the selected equipment. These results are 
also in accordance with what we would expect from compliance with the 3GPP 
Femtocell standards and IEEE 802.11. 

Wi-Fi uses the same frequency channels for their uploading and downloading 
operations. It does this by utilizing the Carrier Sense Multiple Access with Collision 
Avoidance (CSMA/CA) protocol. This protocol is designed to provide fair access to the 
shared channels so that all stations get a chance to use a network (Liqiang, 2011). After 
every packet is transmitted, all stations use this protocol to determine which station gets 
to use the channel next. This process can however slow transmission rates. As shown in 
Tables 7 through 9, and Figures 8 and 9, the results of our baseline testing with the Wi-Fi 
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router is a download average speed of 18.87 Mb/s. The results of our baseline testing 
with the Wi-Fi router is an average upload speed of 4.08 Mb/s. 


Scenario 

Ping (ms) 

Download Speed 
(Mb/s) 

Upload Speed 
(Mb/s) 

Wi-Fi 

25 

18.87 

4.08 

Femtocell 

50.9 

8.06 

0.72 


Table 7. Speedtest.net test averages 


Scenario 

Large 

Packet Ping 

Small 

Packet Ping 

Average 

Packet 

Download 

Speed 

(Mb/s) 

Upload 

Speed 

(Mb/s) 

Wi-Fi 

54.7 

52 

53.2 

14.50 

3.30 

Femtocell 

664.5 

166.7 

415.4 

9.64 

0.62 


Table 8. Ping-test.net test averages 


Scenario 

Packet Loss 

Ping 

Jitter 

Score 

Wi-Fi 

0 

24.9 

6.6 

4.39 

Femtocell 

0 

48.9 

11.3 

4.37 


Table 9. Pingtest.net test averages 


Speedtest.net 


18.87 



4.08 


0.72 


Download Speed Upload Speed (Mb/s) 
(Mb/s) 


■ Wi-Fi 
Femtocell 


Figure 8. Speedtest.net results 
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Ping-test.net 



Download Speed Upload Speed (Mb/s) 
(Mb/s) 


■ Wi-Fi 
Femtocell 


Figure 9. Ping-test results 


Femtocells operate differently than Wi-Fi routers in that they have different 
frequency channels for both download and upload operations (Lopez-Perez, 2008). Since 
these are independent of one another you cannot compare them in the same way as Wi-Fi 
upload and download speeds. In Wi-Fi you simply add the download and upload speed 
averages and get a basic speed. For Femtocells you keep the upload and download 
separate and measure their capacity independently. As shown in Tables 7 through 9, and 
Figures 8 and 9, the results of our baseline testing show a download average speed of 
8.06 Mb/s. The results of our baseline testing show the upload speed average of 0.72 
Mb/s (720 Kbps). 

In terms of basic speed the Wi-Fi’s downloading speed is more than twice as fast 
as the Femtocell’s downloading speed. For uploading operations the Wi-fi is more than 
five times faster. Downloading and uploading speed is not the only place where Wi-Fi 
outperforms the Femtocell, packet transmission time is also an issue. This can be seen by 
our next results in the measurements of ping delay (latency) data. 
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Ping is a measurement of the RTT of a packet of information to its destination and 
back. This includes moving through the Wi-Fi or Femtocell up-link and down-link as 
well as the infrastructure. If you were to assume that the delay in ping only came from 
uploading and downloading values then the Femtocell ping delay should be five times 
longer than the Wi-Fi ping delay. The tests results however showed a ratio of between 
seven and eight. 

A possible reason for this is that the Femtocell has a more complex architecture. 
It has added steps to the process of achieving Internet connectivity. It must first move 
through the mobile operator’s domain and then obtain access to the IP network. These 
steps lead to a larger delay in transit time. This is an additional “operating difference” 
that needs to be considered when evaluating which of the two technologies is more 
efficient. 

Our last area to examine is the jitter results. As jitter is the amount of variance in 
successive ping tests the lower the value the better the connection. In our tests Wi-Fi 
jitter averaged 6.6ms (Table 14) and the Femtocell averaged 11.3ms (Table 15). There is 
not as much of a disparity between these scores, which may suggest that Femtocells tend 
to be slower in communications but can ultimately maintain a stable steady transmission. 

Also of note on our Pingtest.net testing is the overall score rating (Tables 14 and 
15). Pingtest.net gives a Mean Opinion Score (MOS), which is an indication of the 
overall quality of the connection. The MOS ranges from 1 to 5 with 1 being the worst 
and 5 being the best. The MOS score provided by this tool is comprised using an 
algorithm based on the three test components to estimate the connection quality. Tables 
14 and 15 show the MOS results and with a score of 4.39/5 for Wi-Fi, and 4.37/5 for 
Femtocell both are given grades of A (rated as excellent). 
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3 


“Speedtest.net” Upload/Download Test Results 


Test Number 

Ping (ms) 

Download Speed (Mb/s) 

Upload Speed 
(Mb/s) 

1 

24 

16.11 

4.08 

2 

25 

19.24 

4.20 

3 

24 

19.14 

4.15 

4 

24 

19.17 

4.20 

5 

25 

19.27 

4.17 

6 

25 

19.14 

4.15 

7 

25 

19.60 

4.17 

8 

24 

19.16 

4.20 

9 

25 

19.28 

4.16 

10 

25 

18.60 

4.17 

Average 

24.6 ms 

18.87 Mb/s 

4.08 Mb/s 


Table 10. Wi-Fi via router connection results 


Test Number 

Ping (ms) 

Download Speed (Mb/s) 

Upload Speed 
(Mb/s) 

1 

55 

10.21 

0.18 

2 

41 

11.85 

0.35 

3 

55 

12.80 

1.21 

4 

65 

2.13 

0.49 

5 

38 

6.63 

1.12 

6 

45 

13.94 

1.46 

7 

45 

7.90 

1.02 

8 

55 

1.51 

0.38 

9 

55 

11.44 

0.68 

10 

55 

2.18 

0.29 

Average 

50.9 ms 

8.06 Mb/s 

0.72 Mb/s 


Table 11. Femtocell connection results 
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4. 


Results of Ping, Jitter and Packet Loss Utilizing “Ping-test.net” and 
“Pingtest.net” 


Test 

Number 

Large 
Packet Ping 

(ms) 

Small 

Packet Ping 

(ms) 

Average 

Latency 

(ms) 

Download 

Speed 

(Mb/s) 

Upload 

Speed 

(Mb/s) 

1 

56 

56 

56 

11.38 

3.5 

2 

53 

53 

53 

17.06 

3.36 

3 

54 

51 

52 

13.96 

3.3 

4 

56 

50 

53 

13.95 

2.14 

5 

50 

54 

52 

13.71 

3.72 

6 

57 

54 

55 

16.31 

3.76 

7 

57 

50 

53 

14.13 

3.14 

8 

53 

51 

52 

14.61 

3.37 

9 

56 

51 

53 

15.21 

3.37 

10 

55 

50 

53 

14.69 

3.29 

Average 

54.7 ms 

52 ms 

53.2 ms 

14.50 Mb/s 

3.30 Mb/s 


Table 12. Wi-Fi connection Ping-test.net results diagram 


Test 

Number 

Large 

Packet Ping 

(ms) 

Small 

Packet Ping 

(ms) 

Average 

Latency 

(ms) 

Download 

Speed 

(Mb/s) 

Upload 

Speed 

(Mb/s) 

1 

206 

87 

146 

5.64 

0.16 

2 

1478 

119 

798 

10.84 

0.05 

3 

646 

490 

568 

1.73 

0 

4 

1490 

254 

872 

0.84 

0 

5 

1873 

339 

1106 

4.73 

0 

6 

167 

73 

120 

13.40 

0.75 

7 

191 

87 

139 

14.93 

1.14 

8 

227 

74 

150 

14.25 

1.43 

9 

117 

74 

95 

15.36 

1.19 

10 

250 

70 

160 

14.64 

1.48 

Average 

664.5 ms 

166.7 ms 

415.4 ms 

9.64 Mb/s 

0.62 Mb/s 


Table 13. Femtocell connection Ping-test.net results diagram 
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Test Number 

Packet 

Loss (%) 

Ping (ms) 

Jitter (ms) 

Score 
(# out of 5) 

1 

0 

23 

6 

4.39 

2 

0 

36 

12 

4.37 

3 

0 

21 

2 

4.39 

4 

0 

26 

10 

4.38 

5 

0 

27 

11 

4.38 

6 

0 

24 

7 

4.39 

7 

0 

27 

10 

4.38 

8 

0 

21 

2 

4.39 

9 

0 

24 

5 

4.39 

10 

0 

20 

1 

4.39 

Average 

0% 

24.9 ms 

6.6 ms 

4.39/5.0 


Table 14. Wi-Fi connection Pingtest.net results diagram 


Test Number 

Packet 

Loss (%) 

Ping (ms) 

Jitter (ms) 

Score 
(# out of 5) 

1 

0 

61 

21 

4.35 

2 

0 

44 

6 

4.38 

3 

0 

51 

13 

4.36 

4 

0 

46 

6 

4.37 

5 

0 

44 

7 

4.37 

6 

0 

50 

14 

4.36 

7 

0 

51 

15 

4.36 

8 

0 

47 

10 

4.37 

9 

0 

46 

8 

4.37 

10 

0 

49 

13 

4.36 

Average 

0% 

48.9 ms 

11.3 ms 

4.37/5.0 


Table 15. Femtocell connection Pingtest.net results diagram 


B. BASELINE TEST IN REALISTIC ENVIRONMENTS 
1. Introduction 

We established a baseline set of tests and results in ideal conditions. These results 
will be useful as reference points for successive testing. As most users of both Wi-Fi and 
Femtocell technology will be utilizing them in less than ideal settings we now need to 
obtain test results in realistic environments. 
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2 . 


Software Used 


WiEye is a freeware 802.11 Wi-Fi analyzer application for the Android 
smartphone. WiEye is used for wireless site surveys, Wi-Fi scanning, and wireless 
discovery. It displays the name, Basic Service Set Identification (BSSID), channel, and 
frequency of any access points within range. This application can also graph this data to 
evaluate Wi-Fi congestion for each available channel. (http://www.WiEye.net) In our 
research we will be utilizing this application to test the frequency strength in our various 
scenarios. 

OpenSignalMaps is a freeware Android application that allows the user to map 
the location of the tower to which the cellular phone is connected, test the connection 
speed, measure the exact signal strength, and provide a graphical depiction of average 
signal strength over a period of time. 

Xirrus Wi-Fi Monitor is a tool that provides access to information about the user’s 
Wi-Fi environment and the current Wi-Fi connection. Xirrus Wi-Fi Monitor provides 
locations of Wi-Fi networks and their relative distance, wireless settings, Wi-Fi site 
survey, signal strength, IP and MAC addresses, SSID, channel, and security information. 

Speedtest.net is an online performance measuring tool that tests the user’s Internet 
connection bandwidth upload and download capabilities. We utilize this tool to measure 
our capabilities during different scenarios. 

3. Influence of Obstacles 

In less than ideal or realistic environments, obstacles are a significant issue. In 
this research we define obstacles as those things that degrade signal strength and interfere 
with Internet connectivity. In our research we address the following as obstacles; 
distance, non-supporting walls, and supporting walls. Each of these obstacles has a direct 
impact on signal strength and ultimately the total amount of data rate that is achievable. 
For our ideal baseline testing the signal strength for Wi-Fi was -58 dB(mW), and the 
strength for the Femtocell was -76 dB(mW). 
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We know that for both Wi-Fi and Femtocells signal strength is best when it 
travels through open spaces with little to no obstructions between wireless transmitters 
and wireless receivers. Even when there are few obstructions between the transmitter and 
receiver, the overall distance between them also affects the signal strength. Received 
signal power tends to fall rapidly with distance and obstructions; this is known as path 
loss. The capability to enable low signal to noise (SNR) operation is critical to address 
this. Different environments exhibit varying effects of path loss. Indoor path loss is 
typically worse than outdoor path loss due to greater attenuation of dense walls and 
objects (Chandrasekhar, 2008). To best represent these obstacles our research will take 
both obstacles and distance into consideration. 

4. Testing Scenarios 

We have chosen four different testing scenarios to best represent a realistic 
environment. These scenarios vary in distance from the router and take place both 
indoors and outdoors. These testing scenarios also include the addition of obstacles in 
the form of interior and exterior walls. 

In Scenario 1 we test an indoor area 25 feet away from the router with a non¬ 
supporting wall between the router and the testing device. Scenario 2 is the same 
distance of 25 feet but we have moved from indoors to outdoors with the router 
remaining indoors; therefore, there is both a non-supporting and a supporting wall 
between the router and the test device. In scenario 3 we remain outside with both a non¬ 
supporting and a supporting wall between the router and the testing device, but we move 
to 50 feet in distance. In scenario 4 we remain outdoors and continue to have both non¬ 
supporting and supporting walls between the router and the testing device but move 
further away, to a distance of 75 feet. 

Utilizing WiEye, OpenSignal Maps, and Xirrus Wi-Fi monitor, we recorded the 
signal strengths for each scenario: 

Baseline Test: -38 dB for Wi-Fi, and -56 dB for the Femtocell 
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Scenario 1: -71 dB for Wi-Fi (33 dB less than the baseline), and -86 dB for the 


Femtocell (30 dB less than the baseline) 

Scenario 2: -74 dB for Wi-Fi (36 dB less than the baseline), and -88 dB for the 
Femtocell (32 dB less than the baseline) 

Scenario 3: -76 dB for Wi-Fi (38 dB less than the baseline), and -88 dB for the 
Femtocell (32 dB less than the baseline) 

Scenarion 4: -79 dB for Wi-Fi (41 dB less than the baseline), and -94 dB for the 
Femtocell (38 dB less than the baseline) 

5. Baseline Test in Realistic Environments Evaluation 

The results of testing in realistic environments are very interesting. They show a 
better resistance of the Femtocell to both the obstacles and distance. The signal strength 
of the Femtocell seems to also be slightly less attenuated than those of the Wi-Fi. 
Overall, throughout the scenarios Wi-Fi steadily declined in performance while the 
Femtocell actually increased in performance until the final scenario where they both 
dropped in downloading and uploading performance (Tables 16 and 17). 

In order to understand these differences in performance we must start at the 
general carrier frequencies of the two platforms. Wi-Fi systems generally operate at 2.4 
GHZ, and Femtocells operate at 1.8 GHZ. Since the Femtocell carrier frequency is lower 
than that of the Wi-Fi we know that this explains why the Femtocell experienced less 
power loss when compared to Wi-Fi (Chandrasekhar, 2008). 

It can be seen from scenario 1 that a distance of 25 feet and the insertion of a non¬ 
supporting wall have significant effects on the Femtocell signal strength. The baseline 
test recorded -56 dB, and the result of scenario 1 was -86 dB, a difference of 30 dB. Wi¬ 
Fi, too, experienced a significant decrease in signal strength from a baseline of -38 dB to 
-71dB, a difference of 33 dB. 

Comparisons of scenarios 2 and 3 suggest that the difference between inserting 
supporting and non-supporting walls has a greater effect on Wi-Fi than on Femtocells. 
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Wi-Fi experienced additional attenuations for both scenarios where Femtocell results 
stayed the same. 

Zhen, et al. (2011) explains that propagation inside buildings is a serious problem 
for Wireless Local Area Network (WLAN) systems. Zhen goes on to state that 
Femtocells are a popular deployment options for coverage holes and capacity hotspot 
areas, and that “due to the poor outdoor-to-indoor propagation property of in-building 
environment, a dedicated wireless system installed inside the building is often preferred 
for providing indoor users high-data-rate services." Femtocells are cost-efficient 
techniques for this application and results have shown superior performance of the Femto 
system compared to other systems in providing high-data-rate services in most cases with 
a quality-guaranteed scheduler, while the centralized joint scheduling system gives the 
best performance. The centralized scheme can also help improve the system robustness in 
obtaining high performance even in the situation where access points are placed non- 
optimally (Zhen, 2011). 

Scenario 3 suggests a further significant decrease in signal strength when 
transmitter and receiver reach a distance of 75 feet with intervening supporting and non¬ 
supporting walls. Noticeably, the decrease in signal strength is more remarkable for 
Femtocells (an additional 6 dB) than for WiFi (an additional 3 dB). 

In summary, our realistic environment testing has shown that distance and 
obstacles weaken WiFi signals in a very significant way, much more than for Femtocells 
signals. In terms of performance in our realistic environment, we see that Wi-Fi 
download and uploading speeds tend to be more affected by a less than ideal 
environment. In contrast, the downloading and uploading performances of the Femtocell 
are only marginally affected by the obstacles of a realistic environment. The Femtocell, 
therefore, can be more capable in certain realistic environments. This suggests that 
Femtocells mounted on maneuver vehicles may provide better network access potential 
for dismounted personnel inside a neighboring structure than would be provided by a Wi¬ 
Fi access point mounted on the vehicle. 
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Test 

Ping (ms) 

Download 

Speed (Mbps) 

Upload Speed 
(Mbps) 

Signal Strength 
(dBm) 

Baseline 

25 

18.87 

4.08 

-38 

Scenario 1 

26 

14.81 

4.05 

-71 

Scenario 2 

33.1 

15.04 

3.96 

-74 

Scenario 3 

27 

12.05 

3.96 

-76 

Scenario 4 

27 

9.51 

3.86 

-79 


Table 16. Wi-Fi averages for realistic scenario testing 


Test 

Ping (ms) 

Download 

Speed (Mbps) 

Upload Speed 
(Mbps) 

Signal Strength 
(dBm) 

Baseline 

51 

8.06 

0.72 

-56 

Scenario 1 

155 

9.70 

0.60 

-86 

Scenario 2 

467 

11.83 

0.97 

-88 

Scenario 3 

609 

10.66 

1.06 

-88 

Scenario 4 

311 

4.82 

0.53 

-94 


Table 17. Femtocell averages for realistic scenario testing 
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6. 


Scenario 1 Speedtest.net Results 


Test Number 

Ping (ms) 

Download Speed (Mb/s) 

Upload Speed (Mb/s) 

1 

25 

13.86 

4.05 

2 

25 

13.37 

4.17 

3 

25 

18.66 

4.17 

4 

45 

16.18 

4.12 

5 

25 

14.67 

4.12 

6 

25 

13.96 

4.19 

7 

25 

18.53 

4.11 

8 

25 

13.00 

3.86 

9 

25 

14.91 

4.17 

10 

15 

10.96 

3.53 

Average 

26 ms 

14.81 Mb/s 

4.05 Mb/s 


Table 18. Scenario 1 Wi-Fi connection 


Test Number 

Ping (ms) 

Download Speed (Mb/s) 

Upload Speed (Mb/s) 

1 

153 

8.75 

0.57 

2 

159 

8.26 

0.58 

3 

148 

13.55 

0.58 

4 

160 

11.07 

0.56 

5 

159 

9.56 

0.53 

6 

159 

8.85 

0.62 

7 

150 

13.42 

0.49 

8 

159 

7.89 

1.04 

9 

159 

9.80 

0.58 

10 

148 

5.85 

0.42 

Average 

155 ms 

9.70 Mb/s 

0.60 Mb/s 


Table 19. Scenario 1 Femtocell connection 
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7, 


Scenario 2 Speedtest.net Results 


Test Number 

Ping (ms) 

Download Speed (Mb/s) 

Upload Speed (Mb/s) 

1 

35 

13.24 

4.02 

2 

25 

14.36 

4.09 

3 

25 

14.81 

4.04 

4 

46 

14.24 

3.78 

5 

65 

14.98 

3.67 

6 

15 

15.53 

3.93 

7 

55 

17.22 

4.03 

8 

25 

16.27 

3.85 

9 

15 

11.75 

4.03 

10 

25 

18.00 

4.11 

Average 

33.1 ms 

15.04 Mb/s 

3.96 Mb/s 


Table 20. Scenario 2 Wi-Fi connection 


Test Number 

Ping (ms) 

Download Speed (Mb/s) 

Upload Speed (Mb/s) 

1 

735 

13.09 

1.13 

2 

295 

12.67 

1.15 

3 

431 

10.51 

0.77 

4 

497 

10.04 

0.76 

5 

1040 

7.88 

1.17 

6 

315 

15.38 

1.10 

7 

575 

15.53 

0.77 

8 

344 

11.97 

0.77 

9 

162 

7.55 

1.29 

10 

273 

13.70 

0.78 

Average 

467 ms 

11.83 Mb/S 

0.97 Mb/s 


Table 21. Scenario 2 Femtocell connection 
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8. 


Scenario 3 Speedtest.net Results 


Test Number 

Ping (ms) 

Download Speed (Mb/s) 

Upload Speed (Mb/s) 

1 

15 

12.13 

3.83 

2 

45 

13.26 

4.18 

3 

25 

15.67 

3.91 

4 

35 

8.27 

3.40 

5 

25 

15.45 

4.08 

6 

25 

8.54 

4.15 

7 

25 

13.07 

4.04 

8 

25 

11.67 

3.86 

9 

25 

10.16 

4.06 

10 

25 

12.30 

4.11 

Average 

27 ms 

12.05 Mb/s 

3.96 Mb/s 


Table 22. Scenario 3 Wi-Fi connection 


Test Number 

Ping (ms) 

Download Speed (Mb/s) 

Upload Speed (Mb/s) 

1 

431 

14.20 

1.32 

2 

508 

13.77 

1.24 

3 

579 

9.65 

0.64 

4 

174 

16.01 

0.58 

5 

920 

7.41 

1.58 

6 

923 

6.37 

1.32 

7 

581 

10.68 

0.78 

8 

522 

11.57 

1.01 

9 

658 

9.14 

1.34 

10 

268 

7.82 

0.82 

Average 

609 ms 

10.66 Mb/s 

1.06 Mb/s 


Table 23. Scenario 3 Femtocell connection 
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9. 


Scenario 4 Speedtest.net Results 


Test Number 

Ping (ms) 

Download Speed (Mb/s) 

Upload Speed (Mb/s) 

1 

25 

9.54 

3.97 

2 

25 

5.63 

4.07 

3 

15 

7.53 

4.06 

4 

25 

9.56 

3.93 

5 

25 

8.48 

3.85 

6 

25 

8.80 

3.91 

7 

45 

11.97 

4.06 

8 

25 

10.72 

3.53 

9 

25 

10.15 

3.33 

10 

35 

12.68 

3.90 

Average 

27 ms 

9.51 Mb/s 

3.86 Mb/s 


Table 24. Scenario 4 Wi-Fi connection 


Test Number 

Ping (ms) 

Download Speed (Mb/s) 

Upload Speed (Mb/s) 

1 

298 

4.39 

0.52 

2 

323 

2.98 

0.61 

3 

162 

3.92 

0.57 

4 

271 

5.35 

0.51 

5 

340 

3.99 

0.50 

6 

298 

4.05 

0.51 

7 

502 

6.34 

0.61 

8 

271 

5.57 

0.49 

9 

270 

5.68 

0.43 

10 

378 

5.96 

0.51 

Average 

311 ms 

4.82 Mb/s 

0.53 Mb/s 


Table 25. Scenario 4 Femtocell connection 


C. ACCESSING THE INTERNET THROUGH HTTP TESTS 
1. Introduction 

Hypertext Transfer Protocol (HTTP) is a stateless application level protocol that 
is used to transfer data on the Internet. Web browsers and servers exchange information 
in accordance with the rules of HTTP. HTTP is a request/response protocol, which 
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means that a web browser will initiate a request to a server and the server in turn sends a 
response. HTTP deals with every web page and is used in every action involved in 
Internet “web surfing.” 

As HTTP is the foundation of data communication for the World Wide Web 
(http://www.w3.org) it is very important to our research. Our testing will therefore 
address HTTP downloading and point out the differences between the capabilities of Wi¬ 
Fi and Femtocell in terms of downloading data from the Internet. We will also deal with 
the differences between the way Femtocells and Wi-Fi access the Internet. 

2. Software Used 

To address these areas we will use a freeware tool called Downtester. Downtester 
is a tool that tests Internet download speeds in multiple locations throughout the world. It 
allows the user to choose URLs and will systematically test the download speed of each. 
(http://www.nirsoft.net) For this research we have chosen the following six URLs: 
http://www.google.com, http://www.facebook.com, http://www.yahoo.com, 

http://www.baidu.com, http://www.youtube.com, and http://www.cnn.com. 

We will also use the Downtester program to measure the time to download files of 
varying sizes. In our research we will test files that are 20 MB, 50MB, 100MB, and 
200MB in size (see Tables 27 - 34). These sizes were chosen to represent the average 
users downloading needs, as stipulated in Chapter 3. Finally, we will also be using HTTP 
Analyzer, a tool that allows the user to monitor, trace, and analyze HTTP traffic in real¬ 
time (http://www.ieinspector.com/httpanalyzer.com). 

3. HTTP Upload/Download Test Results 


File Size 

Wi-Fi Speed 

(Kbytes/s) 

Femtocell 

Speed 

(Kbytes/s) 

Wi-Fi Speed 

(Mbits/s) 

Femtocell 

(Mbits/s) 

20MB 

888.30 

488.06 

7.28 

3.99 

50MB 

954.54 

524.41 

7.82 

4.29 

100MB 

873.66 

479.98 

7.16 

3.93 

200MB 

413.25 

227.03 

3.39 

1.95 


Table 26. Average results comparing Wi-Fi and Femtocell 

50 






























Table 26 shows our HTTP testing downloading rates that are on average with the 
results from our previous Wi-Fi and Femtocell downloading speed comparison testing. 
Wi-Fi’s downloading speed increases initially and then with the larger size files it falls a 
little at first then it drops significantly. The Femtocells is comparable in its results. It too 
averages approximately 4 Mbits/s until it hits the largest file where it falls to 
approximately 2 Mbits/s. These results show that both Femtocells and Wi-Fi are 
comparable when accessing files of various sizes over the Internet. Only the biggest files 
appear to slow the downloading performances of both Femtocells and Wi-Fi. 

4. Speed Results for 20MB Files 


Test Number 

Speed (Kbytes/s) 

Speed (Mbits/s) 

1 

872.3 

7.15 

2 

839.6 

6.88 

3 

884.6 

7.25 

4 

566.1 

4.64 

5 

880.4 

7.21 

6 

974.8 

7.99 

7 

849.6 

6.96 

8 

951.5 

7.79 

9 

1102.3 

9.03 

10 

961.8 

7.88 

Average 

888.30 KB/Sec 

7.28 Mbps 


Table 27. 20 MB file Wi-Fi connection results 


Test Number 

Speed (Kbytes/s) 

Speed (Mbits/s) 

1 

479.3 

3.92 

2 

461.3 

3.78 

3 

486.0 

3.98 

4 

311.1 

2.54 

5 

483.7 

3.96 

6 

535.6 

4.39 

7 

466.8 

3.82 

8 

522.8 

4.28 

9 

605.6 

4.96 

10 

528.4 

4.32 

Average 

488.06 KB/Sec 

3.99 Mbps 


Table 28. 20 MB file Femtocell connection results 
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5. Speed Results for 50MB Files 


Test Number 

Speed (Kbytes/s) 

Speed (Mbits/s) 

1 

963.3 

7.89 

2 

974.2 

7.98 

3 

929.6 

7.62 

4 

870.1 

7.13 

5 

745.9 

6.11 

6 

1052.2 

8.62 

7 

959.6 

7.86 

8 

1003.9 

8.22 

9 

1091.6 

8.94 

10 

955.0 

7.82 

Average 

954.54 KB/Sec 

7.82 Mbps 


Table 29. 50 MB file Wi-Fi connection results 


Test Number 

Speed (Kbytes/s) 

Speed (Mbits/s) 

1 

529.2 

4.33 

2 

535.2 

4.38 

3 

510.7 

4.18 

4 

478.0 

3.91 

5 

409.8 

3.35 

6 

578.1 

4.73 

7 

527.2 

4.31 

8 

551.5 

4.51 

9 

599.7 

4.91 

10 

524.7 

4.29 

Average 

524.41 KB/Sec 

4.29 Mbps 


Table 30. 50 MB file Femtocell connection results 
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6. Speed Results for 100MB Files 


Test Number 

Speed (Kbytes/s) 

Speed (Mbits/s) 

1 

930.1 

7.62 

2 

947.8 

7.76 

3 

1125.3 

9.22 

4 

944.5 

7.74 

5 

1035.8 

8.49 

6 

873.6 

7.16 

7 

936.4 

7.67 

8 

941.8 

7.72 

9 

627.5 

5.14 

10 

373.8 

3.06 

Average 

873.66 KB/Sec 

7.16 Mbps 

Table 31. 

100 MB file Wi-Fi connection results 

Test Number 

Speed (Kbytes/s) 

Speed (Mbits/s) 

1 

511.0 

4.18 

2 

520.7 

4.26 

3 

618.2 

5.06 

4 

518.9 

4.25 

5 

569.1 

4.66 

6 

480.0 

3.93 

7 

514.5 

4.21 

8 

517.4 

4.24 

9 

344.7 

2.82 

10 

205.3 

1.68 

Average 

479.98 KB/Sec 

3.93 Mbps 


Table 32. 100 MB file Femtocell connection results 
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7. Speed Results for 200MB Files 


Test Number 

Speed (Kbytes/s) 

Speed (Mbits/s) 

1 

674.4 

5.52 

2 

412.4 

3.38 

3 

343.6 

2.81 

4 

393.1 

3.22 

5 

358.2 

2.93 

6 

318.5 

2.61 

7 

336.4 

2.76 

8 

379.6 

3.11 

9 

403.9 

3.31 

10 

512.7 

4.20 

Average 

413.25 KB/Sec 

3.39 Mbps 


Table 33. 200 MB file Wi-Fi connection results 


Test Number 

Speed (Kbytes/s) 

Speed (Mbits/s) 

1 

370.5 

3.03 

2 

226.5 

1.85 

3 

188.7 

1.54 

4 

215.9 

1.76 

5 

196.8 

1.60 

6 

175.0 

1.43 

7 

184.8 

1.51 

8 

208.5 

1.70 

9 

221.9 

1.81 

10 

281.7 

2.30 

Average 

227.03 KB/Sec 

1.85 Mbps 


Table 34. 200 MB file Femtocell connection results 


8. Real Time HTTP Monitoring 

After establishing our basic speeds for downloading various size files via HTTP, 
we now move on to the real time monitoring of HTTP traffic. In this section of our 
research we analyze the process of HTTP traffic. We will start with the components for 
the websites of the six chosen websites. These include Google.com, Facebook.com, 
Youtube.com, Yahoo.com, CNN.com, and Baidu.com. 
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All websites are made up of similar content. “We define content broadly as the 
stuff in your web site.” (Rosenfeld et al, 1998) In general we view data, applications, 
images, and video when we access a website, which are all essentially files; however, the 
way the files are handled and the sensitivity to loss or delay is particular to specific file 
types, based on the requirements of the using applications. As shown in Figures 10 and 
11, our research will break this basic content into percentages of the total amount of 
components in each respective website. We will use this as a reference when we then 
break down the six websites into four phases of the HTTP request. The four phases we 
will cover are the connect phase, content request phase, waiting phase, and receive first to 
last phase. We will then compare the performance of both Wi-Fi and Femtocell in 
download these websites. 

In our tests the connect phase represents the Transmission Control Protocol (TCP) 
level connection time. This is the time that a new TCP level connection is established 
with the web server. The send first to last phase represents the time required to send the 
HTTP request message to the server. This value depends on the amount of data that is 
sent to the server. The wait phase is the idle time taken to receive a response message 
from the server. This value includes network delay time and web server processing time. 
Finally, the receive first to last phase is the time that you receive the response message 
from the web server. This value depends on the size of the return data and the network 
bandwidth. 


55 




Figure 10. Percentage of Wi-Fi accessed website images, text, applications, and video 
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Figure 11. Percentage of Femtocell accessed website images, text, applications, and 

video 
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Figures 10 and 11 show the website profile characteristics based on the 
percentage of total objects present in each of the listed websites. From the information 
provided by Tables 35 and 36, we can see where the biggest areas are and which affect 
performance the most. Images are typically the largest items on most websites, these are 
followed by applications and text. Only one website visited contained video files. 


Wi-Fi 

Images 

Text 

Applications 

Video 

CNN 

48 

35 

17 

0 

Yahoo 

89 

3 

8 

0 

YouTube 

49 

28 

16 

7 

Facebook 

35 

24 

41 

0 

Baidu 

50 

13 

37 

0 

Google 

75 

12 

13 

0 

Average % 

58% 

19% 

22% 

1% 


Table 35. Wi-Fi profile characteristic averages 


Femtocell 

Images 

Text 

Applications 

Video 

CNN 

50 

34 

16 

0 

Yahoo 

89 

3 

8 

0 

YouTube 

50 

28 

18 

4 

Facebook 

35 

24 

41 

0 

Baidu 

50 

13 

37 

0 

Google 

80 

20 

0 

0 

Average % 

59% 

20% 

20% 

1% 


Table 36. Femtocell profile characteristic averages 


Wi-Fi 

Connect 

Send 

Wait 

Receive 

CNN 

19 

1 

48 

32 

Yahoo 

3 

1 

62 

34 

YouTube 

16 

1 

67 

16 

Facebook 

24 

1 

74 

1 

Baidu 

1 

1 

97 

1 

Google 

5 

3 

62 

30 

Average % 

11% 

1% 

69% 

19% 


Table 37. Wi-Fi HTTP breakdown 
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Femtocell 

Connect 

Send 

Wait 

Receive 

CNN 

11 

1 

40 

48 

Yahoo 

13 

1 

43 

43 

YouTube 

41 

1 

49 

9 

Facebook 

15 

1 

83 

1 

Baidu 

45 

1 

53 

1 

Google 

1 

1 

69 

29 

Average % 

21% 

1% 

56% 

22% 


Table 38. Femtocell HTTP breakdown 


Average Beginningto End Times 



■ Wi-Fi 

□ Femtocell 








Figure 12. Average downloading time from beginning to end (in seconds) 


9. Analysis of HTTP Traffic Monitoring Testing 

Dissecting the HTTP process in our testing provides even better insight into the 


issues related to HTTP handling by both Femtocells and WiFi across a number of 

58 













































different sites and applications. Our research shows that the average loading time for a 
page is typically 2 seconds for the Femtocell and slightly less than that for Wi-Fi. These 
speeds are dependent on the page that is accessed. 

As shown in our research the largest component of our selected websites is 
images, at approximately 58% for both Wi-Fi and Femtocell. Applications were the 
second most used component at 20% for both, next is text at 20%, and finally video was 
rarely used and was measured at 1%. 

We next look at the process of accessing a website. The four stages of accessing 
a website are connection, sending, waiting, and receiving. Our research has shown that 
the largest percentage of these four stages was wait time (Tables 37 and 38, and Figure 
12). It was 69% of the total process for Wi-Fi and 56% for Femtocell. The next largest 
percentage was receiving time at 19% Wi-Fi, and 22% for Femtocell. Connect time for 
Wi-Fi was 11% and for Femtocells it was 21%. Finally, the smallest percentage, sending, 
measured at approximately 1%. 

Our research shows us that the difference in loading time is not necessarily 
dependent on the content. The slight difference can be explained by the Femtocells 
slower down-load speed and the additional delay can also be attributable to the long 
mobile operator chain between the Home Node (HNB) and global IP access to the server 
in the case of Femtocells, while WiFi can access directly to the global IP network and 
hence to the server. 


D. STREAMING OF FILES THROUGH RTP TESTS 
1. Introduction 

To test streaming capabilities we utilize Real-time Transport Protocol (RTP) and 

Real-time Transport Control Protocol (RTCP). These protocols provide end-to-end 

network transport functions for applications transmitting real-time data, such as 

interactive audio and video. Using RTP/RTCP we can analyze videos of different sizes 

by using a packet sniffer program. Utilizing a packet sniffer program we can analyze the 

stream for information such as sequence errors, jitter, packet losses, etc. This will 
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provide a comparison between the performances of Femtocells and Wi-Fi while 
streaming a video. 

2. Software Used 

We started with a streaming video test from myspeed.visualware.com that 
measures Real Time Streaming Protocol (RTSP) video and interlaced audio to identify 
high packet jitter and packet loss that causes poor quality video. We then performed tests 
using the packet analyzer, Wireshark. Utilizing Wireshark and a VideoLAN Client 
(VLC) player we selected various size video files (20MB, 40MB, 60MB, and 100MB) to 
stream (http://www.videolan.org). While streaming, we used Wireshark to capture 
packets for analysis. The VLC player provides the option to stream video that has been 
downloaded. We utilized this option to stream these video files of differing sizes via 
RTP to 224.1.1.1 using port 5004. Wireshark was opened and started the capture of 
packets when the video began playing. 

3. Video and Audio Testing 

To start, we ran tests on the quality of the video and audio over IP. This test 
specifically looked at jitter and packet loss. In these results we saw the variance of User 
Datagram Protocol (UDP) RTT over time, providing insight into the jitter experienced. 
The RTT variance must be kept to a minimum otherwise video quality may be degraded. 
In this test, the measure of jitter is the difference in time that each packet takes to reach 
its destination. In an ideal case, each packet sent would take the same time to travel 
between the server and the client. In our tests, we saw that Wi-Fi and Femtocell 
measurements in regards to jitter are very close (Tables 39 and 40). For video, Wi-Fi 
averages 1.54 ms and Femtocell 2.06 ms. Audio jitter is very close, with Wi-Fi averaging 
left audio channel jitter at 1.42 ms and right audio channel jitter at 1.44 ms. Femtocells 
audio jitter measures left channel 1.18 ms and right channel 1.26 ms. In this case the 
Femtocell has less jitter in its audio channels. Our test also compared the amount of 
packet loss between Wi-Fi and Femtocells. This was also very close, with Wi-Fi having 
no packet losses in either left and right audio channels or video. Femtocells had 0.16 
percent loss in all three respective categories (Table 40). 
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The final area addressed in this test was packet order. Packet order is a 
measurement of how many packets arrive in order. Packets do not necessarily take the 
same route or the same time to reach their destinations and when they arrive out of order 
it may lead to delayed or even discarded packets. In our video test Wi-Fi achieved 100% 
packet order and Femtocell 99.82%. Wi-Fi also scored 100% in both left and right audio 
packet order, whereas Femtocell scored 99.82%. 


Test 

Number 

Video 

Jitter 

(ms) 

Video 

Loss 

(%) 

Video 

Packet 

Order 

(%) 

Left 

Audio 

Jitter 

(ms) 

Left 

Audio 

Loss 

(%) 

Left 

Audio 

Packet 

Order 

(%) 

Right 

Audio 

Jitter 

(ms) 

Right 

Audio 

Loss 

(%) 

Right 

Audio 

Packet 

Order 

(%) 

1 

1.5 

0 

100 

1.3 

0 

100 

1.4 

0 

100 

2 

3.2 

0 

100 

3.2 

0 

100 

3.3 

0 

100 

3 

1.6 

0 

100 

1.2 

0 

100 

1.2 

0 

100 

4 

0.7 

0 

100 

0.7 

0 

100 

0.7 

0 

100 

5 

0.7 

0 

100 

0.7 

0 

100 

0.6 

0 

100 

Average 

1.54 

ms 

0% 

100% 

1.42 

ms 

0% 

100% 

1.44 

ms 

0% 

100% 


Table 39. Wi-Fi audio video test 


Test 

Number 

Video 

Jitter 

(ms) 

Video 

Loss 

(%) 

Video 

Packet 

Order 

(%) 

Left 

Audio 

Jitter 

(ms) 

Left 

Audio 

Loss 

(%) 

Left 

Audio 

Packet 

Order 

(%) 

Right 

Audio 

Jitter 

(ms) 

Right 

Audio 

Loss 

(%) 

Right 

Audio 

Packet 

Order 

(%) 

1 

1.3 

0 

100 

1.0 

0 

100 

0.9 

0 

100 

2 

3.9 

0.8 

99.1 

2.0 

0.8 

99.1 

2.3 

0.8 

99.1 

3 

2.3 

0 

100 

1.0 

0 

100 

1.1 

0 

100 

4 

1.4 

0 

100 

0.9 

0 

100 

1.0 

0 

100 

5 

1.4 

0 

100 

1.0 

0 

100 

1.0 

0 

100 

Average 

2.06 

0.16 

99 . 82 % 

1.18 

0.16 

99 . 82 % 

1.26 

0.16 

99.82 


ms 

% 


ms 

% 


ms 

% 

% 


Table 40. Femtocell audio video test 


4. Analysis of Video Streaming Through RTP Test Results 

As videos vary in size depending on the quality and content, we streamed very 


basic videos of average quality and with few special effects and audio variances (teaching 
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and lecture videos). Our 20MB video streamed for ten minutes and, as shown for Wi-Fi, 
there are no packet losses or sequence errors. Utilizing the Femtocell for the same 20MB 
streaming video also reflected zero percent packet loss and no sequence errors. Both 
Wi-Fi and Femtocell show the same results for this size file (Table 41). This is most 
likely due to the low amount of data rate of the video streamed. At this lower level there 
are no packet losses and the downlink channel is not overloaded. This is very close to an 
ideal case scenario for both Wi-Fi and Femtocell. 

When we proceeded to the next set of files, the 40MB streaming video size. At 
this file size the results for Wi-Fi began to show packet losses and sequence errors. Wi¬ 
Fi experienced a 41.95% packet loss and an average of 8.8 sequence errors per file (Table 

42) . While watching the streaming video of this file size you begin to see a slight 
degradation in the quality of the video. The packet losses and sequence errors are having 
an effect on the video but the viewer is still able to watch the video without too much 
frustration. Our results from the Femtocell were still outstanding. The packet losses 
were zero and there were no sequence errors. Streaming the video over Femtocell is 
noticeably smoother than the Wi-Fi at this point. 

Streaming video files of 60MB are approximately thirty minutes in length. Our 
Wi-Fi streaming tests showed a continuing decline. The packet losses increased to 
46.09% and the sequence errors are just under 8 per file. With these levels observed 
more distortion and a few moments of shuttering (or quick stopping and jumping). The 
Femtocell began to experience packet loss and sequence errors streaming this file size. 
The results from the Femtocell were 28.92% packet loss and 3.6 sequence errors (Table 

43) . Here we also observed slight distortion and image shuddering and jumping. 

Videos of 100MB are generally a higher quality 30-40 minute video or a medium 
quality 50 minute video. Our results showed a significant amount of packet loss and 
sequence errors for the Wi-Fi and Femtocell at this size video file. Wi-Fi results reflected 
an average of 81.95% packet loss and 18 sequence errors. With the Wi-Fi video saw 
complete stoppage at several points and distortion. There are some instances with 
missing frames that may be attributed to the high levels of packet loss. For this file size 
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the Femtocell experienced packet losses of 70.79% and 3.8 sequence errors (Table 44). 
The viewing quality is better than the Wi-Fi, but still almost unusable. 

As can be seen by our research Femtocells outperform Wi-Fi in terms of RTP 
video streaming applications. An understanding of the access protocols used by 
Femtocells and WiFi networks provide insight into the possible reason for the difference 
in the performance of these networks for streaming content. The 3GPP release 6 
describes the High Speed Uplink Packet Access (HSUPA) scheme used by Femtocell 
devices; this is likely the key to the Femtocells RTP streaming success. This scheme 
tends to be more efficient in streaming because it utilizes the available spread spectrum 
Dedicated Physical Data Channel (DPDCH), which transports RTP frames very 
efficiently (Hu, et al., 2012). As streaming operates in a continuous fashion channels are 
assigned at the beginning of the stream and these same channels remain used throughout. 
The Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) scheme used 
by WiFi networks is a contention=based access scheme that relies on collision recovery 
mechanism to limit the effect of congestion on the link, but it does not eliminate 
collisions. This scheme is less effective for streaming RTP scenarios as it does not 
dedicate resources to the specific stream. This is due to the fact that Wi-Fi cannot make 
full use of its capacity due to its reliance on contention-based access. This is in part 
caused by Wi-Fi having to move through several protocols this leads to a degradation of 
efficiency throughout the streaming process. In conclusion streaming through RTP 
emphasizes the strengths of Femtocells versus Wi-Fi. Femtocells operate efficiently with 
much less packet loss and sequence errors for streaming video files of various sizes 
(Figure 13). 
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Figure 13. Wi-Fi Femtocell packet loss 


5. Test Results for 20MB Streaming Video 


Test 

Number 

Wi-Fi Packet 
Loss (%) 

Wi-Fi 

Sequence 

Errors 

Femtocell 
Packet Loss 
(%) 

Femtocell 

Sequence 

Errors 

1 

0 

0 

0 

0 

2 

0 

0 

0 

0 

3 

0 

0 

0 

0 

4 

0 

0 

0 

0 

5 

0 

0 

0 

0 

Average 

0% 

0 

0% 

0 


Table 41. 20 MB streaming video results 
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6. Test Results for 40MB Streaming Video 


Test 

Number 

Wi-Fi Packet 
Loss (%) 

Wi-Fi 

Sequence 

Errors 

Femtocell 
Packet Loss 

r%i 

Femtocell 

Sequence 

Errors 

1 

41.69 

9 

0 

0 

2 

40.02 

8 

0 

0 

3 

46.19 

9 

0 

0 

4 

41.07 

9 

0 

0 

5 

40.80 

9 

0 

0 

Average 

41.95% 

8.8 

0% 

0 


Table 42. 40 MB streaming video results 


7. Test Results for 60MB Streaming Video 


Test 

Number 

Wi-Fi Packet 
Loss (%) 

Wi-Fi 

Sequence 

Errors 

Femtocell 
Packet Loss 

f%l 

Femtocell 

Sequence 

Errors 

1 

45.40 

8 

28.88 

4 

2 

40.02 

8 

27.41 

4 

3 

44.90 

7 

30.30 

3 

4 

49.30 

6 

28.55 

3 

5 

50.80 

9 

29.46 

4 

Average 

46.09% 

7.6 

28.92% 

3.6 


Table 43. 60 MB streaming video results 


8. Test Results for 100MB Streaming Video 


Test 

Number 

Wi-Fi Packet 
Loss (%) 

Wi-Fi 

Sequence 

Errors 

Femtocell 
Packet Loss 
(%) 

Femtocell 

Sequence 

Errors 

1 

80.20 

19 

70.79 

4 

2 

78.50 

16 

68.28 

3 

3 

82.11 

20 

71.53 

4 

4 

83.00 

18 

70.02 

4 

5 

86.10 

17 

69.40 

4 

Average 

81.95% 

18 

70.00% 

3.8 


Table 44. 100 MB streaming video results 
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E. VOIP TESTING 


1. Introduction 

Voice over IP (VoIP) is voice delivered using Internet protocol. This means 
sending voice information in digital form in discrete packets rather than in the traditional 
circuit switched protocols used in Public Switched Telephone Networks (PSTN). It is 
generally accepted that the better the upload/download speed the better the connection 
during a VoIP call (Chandrasekhar, 2008). In the case of VoIP, jitter and packet loss are 
significant factors that can lead to poor quality or limited usability. 

2. Software Used 

To perform these tests we used the online testing programs, 
VoIPreview.org/voipspeedtester and myspeed.visualware.com. VoIPreview is a program 
that evaluates the parameters of a VoIP phone call (http://voipreview.org). Myspeed 
provides the calculations of all parameters necessary for a successful VoIP call and 
establishes the evaluation of an all-encompassing Mean Opinion Score (MOS) (Table 45) 
as described in Chapter 3 (http://www.myspeed.visualware.com). The score depiction is 
provided again for ease of reference. The range of 4.0 to 4.5 is generally accepted as the 
level that provides a quality VoIP call. In contrast, any value below 3.5 is considered 
unacceptable. 


Score 

Quality of Communication 

1 

Impossible to communicate. 

2 

Very annoying. Nearly impossible to communicate. 

3 

Annoying 

4 

Fair. Imperfections can be perceived, but sound still 
clear. This is the range for cellular phones. 

5 

Perfect. Like face-to-face conversation or radio reception. 


Table 45. Mean Opinion Score (MOS) 


3. Analysis of VoIP Test Results 

When assessing the differences between Wi-Fi connections and Femtocell 
connections in regards to VoIP we return to the same criteria as our previous tests; jitter, 
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packet loss, and upload/download speed and consistency. Figure 14 shows the VoIP 
requirements for jitter and packet loss. 

When considering jitter with respect to VoIP it needs to be less than 5ms to reach 
standard quality. Our testing shows both Wi-Fi and Femtocells jitter scores are in the 
standard quality range. Wi-fi jitter averaged 6.8 ms and Femtocell jitter averaged 16.45 
ms. While the Wi-Fi scores were better, both are within acceptable VoIP communications 
criteria (http:www.voip-info.org). 

For VoIP applications, packet loss needs to be less than 1% to be considered 
standard quality. Our testing show packet loss for Wi-Fi is 0%, which is perfect. The 
packet loss for Femtocell was very close, measuring 0.06%. Both Wi-Fi and Femtocells 
score in the radio quality category, which is perfect for VoIP communications. 

Packets are very time dependent when it comes to media based applications like 
VoIP. If the packet arrives too late it is discarded and there is a degradation of quality. 
The packet discards experienced during our testing was the first area where Wi-Fi pulls 
noticeably ahead of the Femtocell. In this area where even a slight loss of information 
can noticeably degrade performance Wi-Fi had 0.04% packet discards and Femtocell had 
1.24%. 

With respect to an overall quality rating, as indicated by the assessed MOS score 
(see Table 45) Wi-Fi score was 3.92 and the Femtocell MOS score was 3.54. Both of 
these scores fall in the fair category, where imperfections can be perceived but sound is 
still clear for the most part. 

Lastly our test considered average round trip time, which is related to jitter; the 
quicker the average RTT the better the connection. The average RTT for the Femtocell- 
based connection was longer than for the Wi-Fi. On average, the trip time for Wi-Fi was 
28ms and Femtocells average time was 44ms. This is double the trip time and again is 
due to the longer connection process. However, 44 ms is still well within standard for 
VoIP service (http:www.voip-infor.org). 
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JITTER 


PACKET LOSS 

0 ms 


0% 

1 ms 

<-Radio Quality-> 


2 ms 


0.2% 

3ms 


0.5% 

5 ms 

^-Standard Quality-^ 

1% 



2% 


^-Broken Sound-> 

5% 


<-VolP Unsupportable^ 

50% 

100 ms 


100% 


Figure 14. MyConnection server from 1999-2010 Visualware VoIP chart 

In summary, both Wi-Fi and Femtocell meet the requirements for standard quality 
or better in these VoIP tests. Wi-Fi has a slight, but only negligible edge, over 
Femtocells in terms of performance issues for packet discards and total averaged round 
trip time. 

4. VoIP Test results 


Test 

Number 

Achieved 

Download 

Speed 

(Mbps) 

Achieved 

Upload 

Speed 

(Mbps) 

Download 

Service 

Consistency 

(%) 

Upload 

Service 

Consistency 

(%) 

Average 
Trip Time 

(ms) 

1 

8.97 

4.28 

63 

97 

29 

2 

9.72 

4.25 

96 

97 

29 

3 

8.41 

3.97 

93 

92 

28 

4 

5.96 

4.12 

56 

97 

25 

5 

7.84 

3.85 

72 

96 

29 

Average 

8.18 Mbps 

4.09 Mbps 

76% 

96% 

28 ms 


Table 46. Wi-Fi connection Voipreview.org/voipspeedtester results 
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Test 

Number 

Achieved 

Download 

Speed 

(Mbps) 

Achieved 

Upload 

Speed 

(Kbps) 

Download 

Service 

Consistency 

(%) 

Upload 

Service 

Consistency 

(%) 

Average 
Trip Time 

(ms) 

1 

8.14 

304 

90 

45 

49 

2 

6.78 

259 

56 

21 

38 

3 

7.93 

398 

69 

54 

45 

4 

9.05 

536 

86 

81 

43 

5 

8.24 

193 

90 

56 

45 

Average 

8.03 Mbps 

338 Kbps 

78.2 % 

51% 

44 ms 


Table 47. Femtocell connection Voipreview.org/voipspeedtester results 


Test 

Number 

Jitter 
User to 
Server 

(ms) 

Jitter 
Server 
to User 

(ms) 

Packet Uoss 
User to 
Server (%) 

Packet 

Uoss 

Server to 
User (%) 

Packet 

Discards 

(%) 

MOS 

Score 

1 

3.0 

4.0 

0 

0 

0 

4.1 

2 

3.4 

13.9 

0 

0 

0 

3.9 

3 

3.8 

24.1 

0 

0 

0.2 

3.5 

4 

2.9 

3.8 

0 

0 

0 

4.1 

5 

3.9 

4.3 

0 

0 

0 

4.0 

Average 

3.4 ms 

10.2 ms 

0% 

0% 

0.04% 

3.92 


Table 48. Wi-Fi connection myspeed.visualware.com results 


Test 

Number 

Jitter 
User to 
Server 

(ms) 

Jitter 
Server 
to User 

(ms) 

Packet Uoss 
User to 
Server (%) 

Packet 

Uoss 

Server to 
User (%) 

Packet 

Discards 

(%) 

MOS 

Score 

1 

4.0 

12.2 

0 

0 

0 

3.9 

2 

19.6 

16.6 

0 

0 

4.4 

3.7 

3 

5.5 

52.7 

0 

0 

0.4 

2.8 

4 

7.2 

24.2 

0.2 

0 

0.8 

3.6 

5 

5.8 

16.8 

0 

0.4 

0.6 

3.7 

Average 

8.4 ms 

24.5 ms 

0.04% 

0.08% 

1.24% 

3.54 


Table 49. Femtocell connection myspeed.visualware.com results 
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In this chapter we have performed an in-depth analysis on the capabilities and 
performance of both Wi-Fi and Femtocells. We began by establishing a baseline that 
pointed out the benefits of Wi-Fi’s better data rate over that of Femtocells. However, 
when we performed our testing in less than ideal conditions we learned that Wi-Fi tends 
to suffer more from the effects of obstacles and distance. Our testing on HTTP web 
accessing we learned that Wi-Fi performs better due to the fact that it was built and 
designed for this purpose. Femtocells on the other hand, have a more complicated 
Internet accessing processes that costs it in terms of both time and performance when 
“web surfing.” The testing performed in RTP file streaming showed that Femtocells tend 
to operate more efficiently for larger files than Wi-Fi. Finally our VoIP testing has 
shown that Wi-Fi performs slightly better than Femtocells. We will now move on to 
summarize our testing and provide conclusions as well as recommendations for areas for 
future research. 
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V. CONCLUSIONS AND RECOMMENDATIONS FOR FUTURE 

RESEARCH 


A. CONCLUSIONS 

The purpose of this research was to evaluate the utility of Femtocells as a means 
of providing remote access to smart-phone user where commercial cellular coverage was 
unavailable. This involved comparing the performance and capabilities provided by 
Femtocell enabled networks to that of a traditional wireless Wi-Fi network. To do this 
our research evaluated the performance and capabilities of these two technologies 
through several use-cases, applications, and scenarios. 

Our research began with a series of baseline tests and assessments of the 
performances of these two technologies in near ideal conditions. These tests were 
performed again in less than ideal conditions, containing both obstructions and over 
distances more typical of conditions experienced by deployed, dismounted force elements 
or first responders. In ideal conditions, the baseline tests confirmed that Femtocells do 
extend access to standard cellular systems, but they offer much reduced data rates to a 
user than does Wi-Fi enabled access. This is due to the smaller channel capacity of the 
commercial-off-the-shelf 3G Femtocells. Our field tests provided an interesting insight, 
however, in that the difference in raw data rates was significantly reduced when the 
environment (obstacles and distance) were entered into the equation. When the context is 
in a realistic field environment, the Femtocells deficiencies are significantly reduced as 
compared to those of the WiFi enabled access network under the same environmental 
conditions. 

A potential weakness of the Femtocell is the complicated process of accessing the 
Internet. To access the global IP, Femtocells must go through multiple steps in the 
mobile operator’s domain and consequently each single interaction is more time 
consuming and complex. This complicated process adds time and affects performance. 
This is not the case for Wi-Fi which connects directly to the Internet. The test results 
have shown that this is not a significant time difference it is just a different way of 
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accessing the Internet, as demonstrated by performance tests in the area of Internet 
browsing, or “surfing,” using the HTTP protocol. 

However, our testing of web browsing through HTTP and file streaming through 
RTP indicated very different results between Femtocells and Wi-Fi hosted access 
networks. With HTTP web browsing, Wi-Fi does better than the Femtocell-hosted access 
as it has the larger shared channel capacity. It must be noted, though, that the contention- 
based access mechanism of WiFi hosted networks makes it more susceptible to 
congestion (i.e., several traffic loads associated with increased numbers of users) than 
does a Femtocell-based access network. In RTP testing, the Femtocell performed better 
than Wi-Fi. The complexity and time-consuming Internet accessing process for the 
Femtocell, that is, routing all data traffic through the cellular provider’s core network in 
order to access the Internet, does not appear to be as significant of an issue with 
streaming traffic performance as it is with general web access. RTP streaming is 
essentially a one-way process, with very long sequences that are sensitive to variances in 
packet delays. This process does not penalize the Femtocell-hosted access networks as 
much as it does Wi-Fi due to the shared channel nature of the Wi-Fi-hosted network, 
which is susceptible to interference from neighboring Wi-Fi sources, which degrades the 
respective application performance. Our tests demonstrated this by showing that 
Femtocells operate correctly, with little or no packet loss, for much larger files than WiFi; 
and even under more extreme loads, Femtocells packet losses are half that of WiFi, until 
the point where the communications collapse. 

Finally, we evaluated the VoIP supporting protocols offered by both Femtocell 
and Wi-Fi. Here we found that both Wi-Fi and Femtocell meet the requirements for 
standard or better quality for VoIP operations. Wi-Fi has a slight, but only negligible 
edge, over Femtocells in terms of performance issues for packet discards and total 
averaged round trip time (latency). 

In summary, in baseline testing Wi-Fi provides a better data rate to the user than 

does Femtocells. Wi-Fi, however, suffers more from the effects of obstacles and 

distance. In HTTP web accessing, Wi-Fi excels because it was built and designed for this 

purpose. Femtocells have complicated Internet accessing processes, requiring an existing 
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relationship with a cellular provider. Further, accessing the Internet through the femtocell 
requires redirection through that cellular provider exacerbating latency issues. In RTP file 
streaming Femtocells operate more efficiently for larger files showing less packet losses 
than Wi-Fi. Finally in VoIP testing Wi-Fi performs slightly better than Femtocells. 

The use of smartphones, tablets, and other wireless devices is becoming 
increasingly prevalent and is driving the need for innovations in wireless data 
technologies to provide more capacity, higher speed connections, and higher quality of 
service. Femtocells can provide a useful way for mobile operators to offer a better user 
experience and deliver broadband services indoors consistently and reliably for a 
comparable context of application, distances, and obstacles. 

B. RECOMMENDATIONS FOR FUTURE RESEARCH 

One of the most common issues with Internet connectivity is the effect of multiple 
users on performance. Future research should explore how multiple users connected 
simultaneously could affect the behavior and performance of femtocells and Wi-Fi 
provided connectivity. Researchers could create a scenario where multiple users access 
the network simultaneously and perform tests that monitors traffic through all network 
connections. They could then measure Internet connection bandwidth upload and 
download rates to compare performance and capabilities. 

Future research regarding utilizing the Femtocell in military applications would 
be beneficial. Research would need to be conducted that would create an architecture to 
incorporate the Femtocell in field environments and tests is abilities to perform in 
deployed situations. Issues requiring investigation include policy constraints as well, 
particularly in international contexts. 

Research into the capabilities of utilizing Femtocells as mobile phone range 
extenders in remote areas on bases, in office or barracks, and even on ships would be 
beneficial. 

Finally, in this research we tested Femtocell and Wi-Fi capabilities in both ideal 
and field-like conditions to gain an understanding of their respective capabilities and 
performance. Future research could utilize the same scenarios to tests impacts to HTTP 
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accessing, and streaming via RTP, as well as VoIP, as this research addressed these 
impacts for the ideal environment only. 
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